Self-reported and measured anthropometric variables in association with cardiometabolic markers: a Danish cohort study

Introduction

Suicide is a leading cause of death worldwide, with an estimated 800,000 deaths by suicide occurring annually, or one person dying by suicide every 40 seconds (World Health Organization, 2014). Despite national and international efforts to prevent suicide, suicide rates continue to rise around the world (Alicandro et al., 2019).

To better understand and identify demographic, environmental, psychological, cognitive and neurobiological factors associated with suicidal thoughts and behaviour, we need large samples, as individual factors most likely explain a small proportion of complex phenotypes as suicidal thoughts or behaviours. Large and diverse samples additionally provide the opportunity to study the heterogeneity in associated factors by identifying subgroups. Large-scale international collaborations in consortia for suicide research may provide an important step forward.

One example of these consortia is the Enhancing NeuroImaging Genetics through Meta-Analysis Suicidal Thoughts and Behaviour (ENIGMA-STB) consortium. The aim of ENIGMA-STB is to study the neural correlates of suicidal thoughts and behaviour (STB), by bringing together research groups around the world that have collected both neuroimaging data and assessed STB in individuals with and without mental disorders. These groups use standardized protocols to process their neuroimaging data and then pool data in analyses that have increased statistical power to detect relevant associations.

While these large-scale collaborations have many strengths, it has been challenging to harmonize scores from the different instruments employed to assess STBs across cohorts, and the validity of the findings will depend on the quality of STB measure harmonization. For instance, in our recent large-scale analysis of cortical morphology across 18 research groups within the ENIGMA Major Depressive Disorder (ENIGMA-MDD) consortium (Campos et al., 2020), STBs were assessed using 19 different measures, including single items on STBs from depression severity questionnaires, items from clinical interviews, in addition to items from comprehensive instruments specifically focused on STBs such as the Columbia Suicide Severity Rating Scale (C-SSRS) (Posner et al., 2008; Posner et al., 2011).

To inform future suicide research in international consortia, it is important to examine how these different suicide measures relate to each other and whether they can be used interchangeably. Therefore, the aim of this study was to examine the correlations between the 20 different assessment instruments for STBs used across 27 ENIGMA cohorts. In the first part of this report, we provide an overview of the literature on reliability and validity of commonly-used measures to assess STBs, and the associations between these measures (concurrent validity). In the second part, we present findings from a meta-analysis performed using the measures collected across 27 cohorts within the ENIGMA-MDD and ENIGMA-STB working groups.

Methods

Results

Literature review

An overview of the reliability and validity measures for the different suicide scales and items derived from our literature review are presented in Supplementary Table S1 and S2, respectively. Information on the reliability (inter-rater reliability, internal consistency and test-retest reliability) and validity (concurrent and predictive validity) of these measures was most often available for instruments specifically focused on STBs (e.g., SIS, SSI, C-SSRS), followed by items on suicidal ideation from questionnaires assessing severity of depressive symptoms (e.g., BDI, MADRS, HAM-D). No reliability or validity measures were identified for suicide questions from diagnostic interviews (e.g., CIDI, SCID). The lowest concurrent validity measure identified was between the SIS and C-SSRS scale. Overall, mostly high to moderate correlation or concurrent validity scores (kappa range: 0.3-0.97; r range: 0.22-0.94) between instruments were identified (Figure 1a).

• Download figure
• Open in new tab

Figure 1. Overview of results

a) Literature review results. Reported instrument correlations are shown using an undirected graph. Nodes represent the instruments studied. Edges are coloured based on the average correlation across cohorts, edge width increases with the number of studies from which the correlations were extracted. b) ENIGMA correlation results. Each node represents one of the instruments included in the study. Edge color represents the sample-size weighted average correlation coefficient between two instruments. The thickness of the edge increases with the number of cohorts contributing to estimate the correlation. Generally speaking the thicker the edge the more confidence in the correlation estimate. c) Overview of the instruments used on the enigma analyses and the amount of overlap between them. BDI: Beck Depression Inventory suicidal ideation item; SSI: Scale for Suicidal Ideation; CDRS: Children’s Depression Rating Scale suicidal ideation item; CIDI: CIDI items on suicidal ideation and behaviour; C-SSRS: Columbia Suicide Severity Rating Scale; DIGS: Diagnostic Interview for Genetics Studies items on suicidal ideation and behaviour; HAM-D: Hamilton Depression Rating Scale item on suicidal ideation; IDAS-II: Inventory of Depression and Anxiety Symptoms suicide subscale; QIDS: Quick Inventory of Depressive Symptomatology suicidal ideation item; K-SADS: Kiddie Schedule for Affective Disorders and Schizophrenia suicide items; MADRS: Montgomery-Asberg Depression Rating Scale suicidal ideation item; MINI: Mini International Neuropsychiatric Interview suicidality module; RCADS: Revised Children’s Anxiety and Depression Scale suicidal ideation item; SCID: Structured Clinical Interview for DSM Disorders suicide questions; SIQ: Suicidal Ideation Questionnaire; SIS: Beck’s Suicide Intent Scale; SITBI: Self-Injurous Thouhts and Behaviours Interview; SSS: Suicide Score Scale; YSR: Youth Self-Report suicide item; SIQ-JR: Suicidal Ideation Questionnaire-Junior.

ENIGMA meta-analysis

Sample description and dimensionality reduction

The average age across cohorts was 39 years (SD=16.3). Cohorts had on average 40% male participants. The most commonly available instruments were the suicidal ideation items from the MADRS, HAM-D and BDI questionnaires. Other relatively common instruments included the C-SSRS, QIDS, SSI, and SCID. The majority of instruments were administered by a clinician or trained interviewer, but some self-reported measures were used (Table 2). For complex instruments dimensionality reduction was carried out by extracting common factor scores using factor analysis (see methods). Fit statistics of these models for each complex instrument within each cohort is summarized in Supplementary Table 3. These complex instruments typically measure more than one suicidal construct. For example, the SSI includes sections on protective factors. Thus, a single common factor might not represent the best model underlying the latent structure of these instruments, but it serves our purpose of dimensionality reduction while capturing the main underlying latent factor related to suicidality which these instruments assess.

View this table:

• View inline
• View popup

Table 2.

Instruments used to assess suicidal thoughts and/or behaviour by the different cohorts: self-report and clinician/interviewer administered measures are presented separately.

Correlation analyses

Results of our correlation analyses are summarised in Figure 1b. Full results are provided in Supplementary Table S4. As explained in the methods section, complex instruments were summarised using factor analysis for dimensionality reduction. From now on, when referring to the complex instruments listed in the methods, we are referring to the common factor score obtained by the dimensionality reduction approach. Overall moderate to high correlations (median r=0.44) were observed among all the studied instruments (including single-item and common-factor of complex instruments). Nonetheless, the common factor of the SIS showed poor correlations (median r∼0.20) with most of the instruments tested. This result is not unexpected; the SIS was applied by a single cohort (N=16) and assesses suicide intent during a suicide attempt, and not suicidal ideation or behaviour as the other instruments do (see discussion).

The instrument with the highest consistency (i.e., highest median weighted correlations with other instruments) was the IDS-Clinician rated measure (median r=0.76). However, few pairs of cohorts had data for this instrument. The C-SSRS and SSI instrument showed a very high concordance (r=0.83; N=191) with each other. In addition, there was a strong correlation between the HAM-D suicidal ideation item and the same item in the version of this questionnaire for children, the CDRS, but this was supported by a single cohort (r=0.94, N=20). The MADRS suicidal ideation item showed a high correlation with the HAM-D (r=0.67, N=1,087) and BDI (r=0.74, N=844) suicidal ideation items and with the SSI instrument common factor (r=0.67, N=119). The HAM-D and BDI suicide items showed only a moderate correlation (r=0.41, N=2,555) between them. Both of these measures were moderately correlated with the SSI (r=0.38, N=429 and r=0.36, N=350 respectively). Moderate to low correlations were observed for the group comprising child scales (YSR, RCADS and CDI), but these were only collected by one cohort. The MINI and SSS common factors showed a low correlation (r=0.12), which was also supported by a single cohort only (N=64).

Sensitivity analysis: recent versus lifetime STB

Cohorts applied different instruments assessing STB with different time frames. For example, the C-SSRS can be used to assess lifetime, time since last assessment, and recent (past 2 weeks) suicidal behaviour information, whereas other instruments might be worded around the past month, two weeks, week or even at the time of assessment. This is a potential source of heterogeneity for studies wishing to compare across these measures. For this reason, we repeated the analyses only focusing on measures applied to a recent (up to past month) time frame, and compared them to the results shown above. These analyses showed similar correlations overall. Notably, concordance between the C-SSRS and the HAM-D, as well as C-SSRS and BDI, were higher when focusing only on recent instruments (Figure 2a-b and Supplementary Table S5).

• Download figure
• Open in new tab

Figure 2 Sensitivity results

Undirected acyclic graph shows the results for the meta-analysis of correlations of suicide risk assessment instruments across ENIGMA cohorts for the complete results (a); using only measures assessing recent (up to past month) suicidal behaviour (b); instruments administered via self-report (c) or using clinician or interviewer based instruments (d; see Table 2). Each node represents one of the instruments included in the study. Each edge color represents the sample-size weighted average correlation coefficient between two instruments. The thickness of the edge increases with the number of cohorts contributing to estimate the correlation. Generally speaking the thicker the edge the more confidence in the correlation estimate. *For complex instruments dimensionality reduction was carried out by extracting common factor scores.

Comparing clinician and self-report scales

We gathered information on whether the distinct instruments were administered by a clinical interview or by self-report (Table 2), and the majority of instruments were administered by a clinician or trained interviewer. As a secondary sensitivity analysis, we repeated the analysis by including only either self-report or interviewer administered instruments (Figure 2c-d). Within interviewer-administered scales, high correlations (r>0.7) were observed between the SSI, C-SSRS, MADRS and QIDS instruments. A similar result was observed between HAM-D and MADRS (r=0.67). The SIS still showed a lower consistency with most other instruments (Figure 2c). For the self-report based instruments, less data was available. Among the self-reported instruments, the BDI and IDAS as well as the SSI and QIDS instruments showed a high concordance (Figure 2d). In general, higher correlations were seen for the interviewer-based measures (see Supplementary Table S6 for the summary of the results). For interviewer-based measures, we were able to replicate the pattern from the main analyses: high correlations between single-item measures and measures assessing recent suicidal ideation (HAM-D, MADRS), strong correlations between detailed measures of STB (C-SSRS and SSI) and low correlations between the SIS and other measures. Nonetheless, we identified pairs of instruments such as the HAM-D and BDI whose low correlation in the main analysis might be explained by differences in administration (i.e., self-report vs interviewer).

Discussion

Our study is a comprehensive assessment of how well different suicide risk assesment instruments relate to each other the extent to which they can be used interchangeably. Harmonization reduces heterogeneity and increases power for discovery analyses, but also enables the assessment of the generalisability of studies and opens up the opportunity to investigate other aspects such as interactions and individual variation analysis (van Harmelen et al., 2020). Identification of correlates of suicide risk may be improved by increasing sample sizes, and by pooling data across studies to detect small effect sizes, which may result from large variance in underlying mechanisms. Our study aimed to examine the concurrent validity of instruments commonly used to assess STBs. To this end, we compared individual level responses across questionnaires by pooling data from 27 cohorts belonging to the ENIGMA-MDD and ENIGMA-STB working groups. We compared our results to a systematic literature search across 180 studies.

A potential source of variance is the heterogeneity introduced by using different suicide risk assessment instruments that each measure slightly different underlying phenomena. Both the results of our analysis and our literature search identified moderate to high correlations between the most commonly used instruments to assess suicidal ideation including the BDI, SSI, HAM-D and MADRS, and between complex instruments (such as the C-SSRS and SSI). These findings are consistent with another study which showed strong correlations between the SSI, BDI and HAM-D (Desseilles et al. 2012). Nonetheless, our results were consistently more conservative than the literature (e.g., showing a lower degree of correlation). This could be explained in several ways. First, this might be evidence for publication bias whereby only positive and expected associations are published. Finally, heterogeneity arising from the way these instruments are administered (i.e. time frames for suicidal behaviours and self-report vs interview based) might also affect the results. We performed sensitivity analyses testing for these factors. For example, we proposed the low correlations between BDI and HAM-D may be explained by the fact that they are typically administered via self-report and a clinician interview respectively. There are studies reporting that the discrepancy between subjective and objective depression severity predicts differential treatment response in bipolar disorder (Suzuki et al., 2016). It is unclear whether a similar phenomenon underlies our results, given that the discrepancy is seen across cohorts rather than in individuals. Another example includes the correlation between the BDI and CSSRS common factor, which became higher when focusing on the interview based cohort only. Nonetheless, the heterogeneity explanation does not always hold true, for example, the MADRS (also interview-based) correlated well with the BDI.

Our unique methodology was chosen to integrate slightly different versions of instruments (i.e., self-reported vs. interview administered). By extracting common factor scores for the complex instruments, we were able to obtain a measure of the underlying suicidal liability being measured by the instruments while reducing the need to adjust for slightly different wording between versions. This approach might be more conservative than those used in previous studies as it is focused on measuring suicide liability rather than a specific construct such as attempt or ideation. As such, instruments that mainly focused on a specific aspect of STB such as suicidal ideation (for example the SIQ) are expected to show a lower correlation with more broad instruments that assess a range of relevant behaviors, such as the C-SSRS.

It is worth noting that suicidal constructs may share a common liability, but they may also have partially independent aetiologies. The Suicide Intent Scale (SIS) was the only measure with overall low measures of consistency in both our analyses and the literature search. However, caution is warranted in interpreting these findings as they were based on two cohorts only, which collected data on the SIS. Although our literature search identified a low concordance between the SIS and both the SIQ and the C-SSRS, a high concordance was reported between SIS and the K-SADS. These results are likely explained by the fact that the SIS is a questionnaire mainly focused on assessing intent of a past suicide attempt. This is further complicated by the intricate relationship between suicidal ideation, attempt, and actual suicide intent. Participants might engage in a suicide attempt with a relatively low intent to die. In fact, previous studies have identified that combining the SIS with other scales increases sensitivity and specificity for predicting suicide deaths (Stefansson et al., 2015).

Our study represents a comprehensive approach to assess the concordance and reliability of commonly used suicide risk assessment instruments. Nonetheless, some limitations need to be considered when interpreting our findings. Our literature search was as exhaustive and systematic as was practicable; however, we cannot rule out the possibility that some relevant studies were excluded because they are not indexed in the databases we searched. Language and cultural differences between cohorts might also affect whether two instruments are concordant. Our study comprised predominantly English speaking participants, but some cohorts included French, Dutch and German speaking participants. Limited research is available on whether language affects reporting of psychiatric symptomatology (Erkoreka et al., 2020). Participants might have undergone evaluation at different points in time in relation to the timing of suicidal behaviors or thoughts, and recall bias could lower the concordance between instruments. In fact, our sensitivity analysis showed that the time frame of the scale used can affect how different measures compare with each other. When focusing on complex instruments, we performed dimensionality reduction using factor scores derived from full-information maximum likelihood. Our approach was based on the fact that some instruments have multiple versions. Thus, our approach is not an exploratory or confirmatory factor analysis of the complex instruments used here. Performing such a study is outside the scope of this manuscript as it would require complete harmonisation of the questionnaires across cohorts and to focus solely on the complex instruments.

While our sensitivity analysis only distinguished between lifetime and recent time frames, it is possible that these effects exist even within recent time frames such as instruments assessing current vs. past two week behaviours. These limitations might explain the lower concordance identified by our analysis compared to the literature synthesis, which studied a compendium of smaller albeit less heterogeneous studies. Finally, while this was a large study, many sites had a distinct combination of measures collected (max N∼ 2,500); therefore, we did not have the power to perform additional sensitivity analyses in adults or adolescents only. Future studies should focus on addressing this, as there are clear factors associated with suicidality (e.g., mood reactivity) that are more prevalent during adolescence (Armey et al., 2015). We were also unable to stratify our analyses by type of psychiatric diagnosis, while prior work found that the correlation between self-report and clinician-reported suicidal thoughts may differ across disorders (Gao et al., 2015; Kaplan et al., 1994; Perugi et al., 2019).

Overall, our results suggest that the most commonly used instruments show a moderate to high concordance. Use of different measures of suicidality, might increase heterogeneity depending on the distinct dimensions and constructs assessed by each instrument. Our study could enable the implementation of composite scores by weighing more concordant measures heavily and penalising less concordant measures. In the absence of such an approach, large-scale collaborations could focus on strictly defined suicide constructs such as suicidal ideation, attempt, and intent that are preferentially defined using the most common instruments such as - amongst others - the HAM-D, MADRS, SSI, C-SSRS. In the absence of a common instrument, or when using an instrument that we found to have low concordance, sensitivity analyses could be performed to assess whether significant results are robust, or at least consistent, after excluding cohorts using the least common instruments. Future studies that plan to collect data on suicidal thoughts and behaviours would benefit from including one or more of the instruments that showed strong correlations with instruments such as the MADRS, SSI, and C-SSRS.