ABSTRACT
Background A rapid review of systematic reviews was conducted to assess the effectiveness of digital mental health interventions for people with a chronic disease. Although it provided an overview of the evidence, it offered limited understanding of ethe types of interventions that were the most effective. The aim of this study was to perform a meta-analysis of primary studies identified in this rapid review of systematic reviews by focusing on the needs of knowledge users.
Methods This secondary meta-analysis follows a rapid review of systematic reviews, a virtual workshop with knowledge users to identify research questions and a modified Delphi study to guide research methods. We conducted a secondary analysis of the primary studies identified in the rapid review. Two reviewers independently screened the titles and abstracts and applied inclusion criteria: RCT design using a digital mental health intervention in a population of adults with another chronic condition, published after 2010 in French or English, and including an outcome measurement of anxiety or depression.
Results 708 primary studies were extracted from the systematic reviews and 84 primary studies met the inclusion criteria Digital mental health interventions were significantly more effective than in-person care for both anxiety and depression outcomes. Online messaging was the most effective technology to improve anxiety and depression scores; however, all technology types were effective. Interventions partially supported by healthcare professionals were more effective than self-administered.
Conclusions While our meta-analysis identifies digital intervention’s characteristics that are more effective, all technologies and levels of support can be used considering implementation context and population.
Review registration The protocol for this review is registered in the National Collaborating Centre for Methods and Tools (NCCMT) COVID-19 Rapid Evidence Service (ID 75).
BACKGROUND
Chronic diseases are the main burden on health systems in developed countries and account for almost 70% of deaths worldwide (1). The majority of people with a chronic disease have more than one concurrent condition and are also at higher risk for developing comorbidities in mental health, including anxiety and depression (2). The prevalence of depression together with another chronic health disease is an important and well-documented phenomenon (3). Studies have also reported a higher prevalence of anxiety disorders in relation to different chronic diseases (4). Increased use of health services is also observed in people living with a chronic disease and with a concomitant depressive disorder (5). Canadians primary healthcare interdisciplinary teams, surveyed during the COVID-19 pandemic, reported a need for broadening services offering to answer the increase in encounters for mental health issues (6). There is an urgent need for more relevant and accurate data on digital interventions in this area to prepare for an increase demand for mental health services.
Practice-based interventions in primary care settings have been shown effective to improve the management of depression in people with chronic diseases (3). In addition, a large number of interventions using digital technologies have been evaluated for the management of depression or anxiety (7, 8), and systematic reviews indicate that they are effective in providing timely and delocalized care in some populations (9). However, it is still unclear what elements or characteristics of digital interventions for mental health are effective (9).
A rapid review provides knowledge users with data that can be readily used to inform healthcare decisions (10). When the topic is broad, it can lead to data that lack precision to fulfil knowledge users’ needs. In an effort to gather data on the effectiveness of digital mental health interventions for people with a chronic disease, a rapid review of systematic reviews was completed (11) and offered only an overview of the problem. The aim of this study was to conduct a secondary meta-analysis of the data collected in this rapid review of systematic reviews by using a knowledge-user panel to guide research methods.
METHODS
We engaged with a panel of knowledge users (patients, clinicians, decision makers), content experts, review methodologists, and researchers throughout the review process, including question development, literature search, interpretation and writing of results, and dissemination of findings.
Preliminary research
This secondary analysis is based on the data from a rapid review of systematic reviews (11). We followed guidelines outlined by the Cochrane Handbook chapter regarding Overview of Reviews and the Cochrane Rapid Reviews Methods Group (12, 13). The review identified a large body of evidence (35 systematic reviews) showing that digital mental health interventions were effective and safe for people with chronic diseases and cancer but that the evidence was still lacking for children and youth populations. In order to inform the knowledge users at each step, the first stage and lessons learned while developing the project were published elsewhere (11, 14).
Three research activities followed the rapid review: a virtual workshop with knowledge users to present our preliminary results and gather their suggestions, a modified Delphi study to prioritize the proposed suggestions for the next stage of the review, and a secondary analysis of the primary studies identified through the rapid review. All activities of the study are summarized in Figure 1.
Summary of the activities and timeline
Workshop
A total of 10 knowledge users, from provincial (Quebec Ministry of Health and Social Services, Institut national d’excellence en santé et services sociaux), regional (Integrated Health and Social Services Centers) and local organizations were invited to take part in a virtual meeting on July 16th 2020 where the preliminary results from the rapid review were presented and discussed. During the workshop, knowledge users were invited to share their experience regarding digital health interventions for mental health issues, in particular in the context of the COVID-19 pandemic. They were welcome to ask questions and to comment on the findings of the rapid review and its relevance for their practice. They were also encouraged to identify knowledge gaps that could be addressed in the next stage of the review.
Delphi Study
Following the workshop with knowledge users, a summary of the main knowledge gaps identified was performed by the research team. We translated these knowledge gaps into nine review questions that were used as the basis for a modified Delphi study. We developed a questionnaire using the RedCap system (15), and sent a personalized invitation to participants in the workshop, inviting them to complete the survey. These knowledge users were also invited to suggest names of potential additional knowledge users who could have an interest in the topic. The questionnaire comprised two sections. First, participants had to rate on a 5-point Likert scale the importance, relevance, and applicability in the context of COVID-19, each of the nine potential review questions. Second, participants were invited to rank each of the nine questions (1 = most important; 9 = less important) according to their preference. A total of 16 knowledge users were invited to take part in a two-round Delphi process in order to identify the key question for the next stage of the knowledge synthesis. All the knowledge users completed the two rounds and all questions reached consensus using median and IQR. The prioritized question after the second round was: “What types of digital health interventions (according to a recognized categorization) are the most effective for the management of concomitant mental health and chronic disease conditions in adults?”
Secondary Analysis
To provide an answer to knowledge users, we conducted a secondary analysis of the primary studies included in the reviews identified in our previous rapid review, published elsewhere [1]. Based on the input from knowledge users, we added inclusion criteria at this stage. Thus, we included only randomized-controlled trials (RCT) presenting a digital health intervention for the management of concomitant mental health and chronic disease conditions in adults, published during the last 10 years (since 2010) in French or English, and including an outcome measurement of anxiety and/or depression. We focussed on anxiety and depression because it was the most prevalent outcomes to answer the question of knowledge users.
All the primary studies referenced in the 35 systematic reviews were extracted and included for the citation screening. Six reviewers individually performed screenings for titles, abstracts and then full text using pilot-tested standardized forms (25 citations for the first level of screening). All citations were reviewed by two reviewers independently at the first level of screening. A standardized extraction form was developed that included study characteristics (e.g., authors, country, design), intervention characteristics (e.g., type of digital intervention), and outcomes reported (measurement tools used, means, standard deviation and time of measurement). Data were extracted by four research associates, and a senior investigator (MPG) completed a quality appraisal of all extracted data. Information related to the study characteristics (first author, date of publication, country, population, health condition, type of intervention, outcomes measured, means, standard deviations, author’s conclusions) were extracted directly in the DistillerSR tool (16).
As knowledge users were interested in obtaining evidence on specific types of digital health interventions for concomitant mental health and chronic health conditions, we looked for existing classifications of digital health tools to sort interventions. We consulted the WHO Classification of digital health interventions (17) and previous reviews on digital mental health solutions (18, 19). Given the limitations of existing categorizations, we used our own system which considered the technology system, the synchronous or asynchronous nature of the intervention, and the level of professional support (self-administration, partially guided, guided). The classification used ten technology systems categories (not exclusive): Internet or Website, Computer software, Mobile application, Electronic messaging (email, SMS), Electronic health record, Telehealth (telemedicine, telepsychiatry), Virtual reality/ augmented reality, Robot, Connected devices and Other system.
We completed a meta-analysis of the standardized means difference (SMD) with an analysis of heterogeneity (x2 and I2) for the two outcomes of interest. We used Cohen D, fixed effects meta-analysis and the R software for data analysis.
The Cochrane Risk of Bias for Intervention Studies (ROBIS) tool was completed by two investigators (MS, MPG) to assess the probability of bias in the included studies. Five types of biases were considered: 1) Risk of bias arising from the randomization process; 2) Risk of bias due to deviations from the intended interventions; 3) Missing outcome data; 4) Risk of bias in the measurement of the outcomes; and 5) Risk of bias in the detection of the reported results. An overall risk of bias was also assessed.
We report our results based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement (20).
RESULT
Characteristics of Included studies
The flow diagram of studies included in the secondary analyses is presented in Figure 2. All individual primary studies included in the systematic reviews of the rapid review were considered.
Forest plot: Subgroup analysis of self-directed interventions for any digital intervention vs. usual care or another digital intervention to manage anxiety in people with any concomitant chronic condition.
A total of 708 primary studies were identified from the systematic reviews included in the rapid review. A total of 429 primary studies were excluded at title & abstract screening stage (duplicates: 91; publication year or study design: 338). Following screening of full text, we excluded 195 records with reasons (Population: 40; Intervention: 74; Outcomes: 67; Language: 2), resulting in a total of 84 primary studies included in the secondary meta-analysis.
Characteristics of Included Studies
We included 84 primary studies, published between 2010 and 2019. Those studies comprised a total sample of 11037 participants. Countries where the studies were conducted included: Sweden (23), the United States (18), Australia (14), Netherlands (9), United Kingdom (7), Germany (6), Switzerland (2), Norway (1), Canada (1), Jordan (1), New Zealand (1) and South Korea (1).
Most studies described interventions performed in the community (33%) and targeted a mixed gender adult population (91%). All studies evaluated digital interventions to manage and treat mental health issues, and a majority (80%) were based on cognitive behavioural therapy (CBT). Most studies compared digital health interventions to usual care (76%), although some studies compared two or more digital interventions (24%). The complete description of included studies is presented in Table 1.
A summary of the estimated effect size for each intervention characteristics, heterogeneity and inconsistency for all comparisons, and for both outcomes are presented in Table 2.
Summary of comparative intervention caracteristics
Global mean differences
The first analysis aimed at the global group differences of any digital intervention compared to usual care or another digital intervention to manage anxiety or depression for people with any concomitant chronic condition.
A total of 62 studies including 8719 participants with anxiety outcomes measures were included in the meta-analysis of the overall effect of digital interventions on anxiety outcomes (additional file 1). The results showed a significant decrease in the anxiety score related to digital health interventions compared to usual care or another digital intervention [Standardized Mean Difference (SMD) = -0.40; 95% Confidence Interval (CI) = -0.35; -0.44)]. Although heterogeneity is high between studies (I2 = 85.77%) the results are relatively consistent across studies (Table 2).
Regarding depression outcomes, we conducted a meta-analysis including 75 studies with 9970 participants. Figure 4 shows a significant reduction in depression scores associated with the use of digital health interventions compared to usual care (SDM = -0.33; 95% CI [-0.29,-0.37]). Heterogeneity is also high for this overall comparison (I2 = 76.35%) with results consistent across studies (Table 2).
Forest plot: Subgroup analysis of partially guided interventions for any digital intervention vs. usual care or another digital intervention to manage anxiety in people with any concomitant chronic condition.
Level of professional support
The forest plots for comparing the level of professional support are presented for anxiety (figures 3 and 4) and depression (figures 5 et 6) outcomes separately.
Forest plot: Subgroup analysis of self-directed interventions for any digital intervention vs. usual care or another digital intervention to manage depression in people with any concomitant chronic condition.
Forest plot: Subgroup analysis of partially guided interventions for any digital intervention vs. usual care or another digital intervention to manage depression in people with any concomitant chronic condition.
Completely self-administered was the delivery method used in 30 studies (5312 participants) with anxiety outcomes and 40 studies (6379 participants) reporting depression outcomes. Self-administered delivery showed a significant decrease in anxiety scores [SMD -0.35 (−0.30, -0.41)] and depression scores [(SMD -0.28 (−0.23, -0.33)].
Partial support by a healthcare provider was used in 29 studies (3206 participants) reporting anxiety outcomes, and 33 studies (3470 participants) reporting depression outcomes. Partial support showed a significant decrease in anxiety [SMD -0,46 (−0.39, -0.53)] and depression scores [SMD -0,43 (−0.36, - 0.50)].
Interventions entirely guided by a healthcare professional were used in three studies (201 participants) reporting anxiety outcomes and two studies (121 participants) reporting depression outcomes. Interventions entirely supported by healthcare professionals showed a significant difference for anxiety scores [SMD -0.46 (−0.16, -0.76)] but no significant difference between groups for depression scores [SMD 0.15 (0.53, -0.21)].
Type of technology
Eight different technologies were used in studies reporting anxiety outcomes. and half had more than one study on which we could perform a meta-analysis (figure 7 and 8). For studies reporting a depression outcome, seven technologies were used, and five of them had more than one study (figure 9 and 10). Forest plots that include all studies are presented in additional file 3 and 4.
Forest plot: Subgroup analysis of electronic messaging interventions for any digital intervention vs. usual care or another digital intervention to manage anxiety in people with any concomitant chronic condition.
Forest plot: Subgroup analysis of internet or website interventions for any digital intervention vs. usual care or another digital intervention to manage anxiety in people with any concomitant chronic condition.
Forest plot: Subgroup analysis on electronic messaging interventions for any digital intervention vs. usual care or another digital intervention to manage depression in people with any concomitant chronic condition.
Forest plot: Subgroup analysis on internet and website interventions for any digital intervention vs. usual care or another digital intervention to manage depression in people with any concomitant chronic condition.
Electronic messaging was used in 23 studies (2700 participants) reporting anxiety outcomes, and 25 studies (2915 participants) reporting depression outcomes. Electronic messaging showed significant improvement in anxiety scores [SMD -0.48 (−0.39, -0.56)] and depression scores [SMD -0.48 (−0.40, - 0.56)].
Internet and website technologies were used in 58 studies (8305 participants) reporting anxiety outcomes, and 70 studies (9492 participants) reporting depression outcomes. Internet and website interventions showed significant improvement in anxiety [SMD -0.39 (−0.35, -0.44)] and depression scores [SMD -0.33 (−0.29, -0.37)].
Telehealth and telemedicine were used in four studies (394 participants) reporting anxiety and/or depression outcomes. Telehealth and telemedicine showed significant improvement in anxiety [SMD -0.50 (−0.29, -0.70)] and depression scores [SMD -0.75 (−0.54, -0.96)].
A computer software was used in two studies (114 participants) reporting anxiety outcomes and four studies (191 participants) with depression outcomes. Computer software showed significant improvement in anxiety [SMD -0.57 (−0.18, -0.96)] and depression scores [SMD -0.55 (−0.26, -0.85)].
Mobile applications were used in one study (76 participants) with anxiety outcomes and two studies (196 participants) with depression outcomes. Mobile applications showed significant improvement for anxiety scores [SMD -0.52 (−0.04, -1.00)], but no significant difference between groups for depression scores [SMD-0.26 (0.02, -0.55)].
Connected devices were used in one study with 51 participants and showed no differences between groups for anxiety [SMD -0.22 (0.33, -0.77)] and depression scores [SMD -0.13 (0.42, -0.68)]
Virtual reality was used in one study with 80 participants with anxiety outcomes and showed significant improvement on anxiety scores [SMD -1.73 (−1.21, -2.25)].
One other type of technology in the form of Digital Video Disk (DVD) was used in one study (220 participants) with anxiety outcomes and two studies (273 participants) with depression outcomes. DVD interventions showed no differences between groups for anxiety [SMD -0.15 (0.11, -0.42)] and depression scores [SMD -0.10 (0.14, -0.34)].
Risk of Bias
Figure 11 presents the risk of bias across studies for each domain. Most of the included studies showed an overall low risk of bias. However, risk of bias was generally high for the Domain 4: Risk of bias in the measurement of the outcomes. In fact, blinding of study participants was not done in most studies and outcomes were self-reported, leading to a high risk of performance bias. This bias is present across studies and would eventually lead to an overestimation of the effect.
Assessment of the risk of bias across studies
DISCUSSION
This knowledge synthesis aimed to rapidly provide evidence for knowledge users regarding the types of digital mental health interventions that were the most effective for people living with a concomitant chronic disease. This secondary analysis answers a specific research question based on knowledge users’ needs and prioritization. Thus, preliminary work in the form of a workshop and a two-round Delphi study were conducted to identify the top-priority question for knowledge users. This question was: “What types of digital health interventions are the most effective for the management of concomitant mental health and chronic disease conditions in adults?”. A total of 84 primary studies including anxiety and depression outcomes were identified from the systematic reviews.
Overall, the results show that digital health interventions are effective to manage mental health issues in adults living with a concomitant chronic condition. The magnitude of the effect varies for anxiety and depression, and heterogeneity is generally high, but the effect size and direction are consistent across studies. Subgroup analyses show that digital mental health interventions with partial support from a healthcare provider have a larger effect size than self-administered interventions. These results are in line with what is known for other populations (21, 22). There is not enough evidence to conclude in the effectiveness of digital interventions that are completely guided by a healthcare professional because of the lack of studies in this category. One key challenge of self-administered digital mental health interventions is sustaining engagement and reducing dropout (23, 24). Partially supported interventions are mitigating these challenges by improving interactivity and personalization (25, 26). However, it is also documented that patients could prefer to only interact with a platform instead than talking with a healthcare professional, highlighting a need for flexible interventions (27).
Regarding the type of technology used, our analyses show that the most effective type of intervention is electronic messaging, but that all types of technologies are effective for both anxiety and depression scores. This finding adds to the literature on the use of digital mental health to reduce disparities. Indeed, as our results show that all types of technologies are equally or more effective than usual care, stakeholders could choose and implement interventions in relation with the needs of the population. For example, decision makers can tailor their choices with respect to cost, ease of access or easing stigma barriers (28).
More research will be needed for newer technologies, such as mobile apps and virtual reality, which have showed effectiveness only in a small number of studies with a large confidence interval.
The significant heterogeneity observed between studies in every comparison is not surprising and could be likely due to differences in comorbidity, outcome measure used, and content of the intervention. However, patterns shown in this meta-analysis are useful for clinical use and implementation.
This knowledge synthesis was informed by knowledge users in order to validate the review questions considering their needs, and identify knowledge gaps that would require more evidence. We used a two-stage process, starting with a rapid review of systematic reviews followed by a secondary analysis of the primary studies. Although we used a systematic approach for selecting these studies, a major limitation is that more recent studies were not included in the analyses. In order to meet the requirement of the funding agency and the urgent need for evidence in the current pandemic, we considered only the most recent studies (published from 2010) from the included reviews. We also assessed the risk of bias in the included studies.
Results from the meta-analyses should be interpreted with caution since heterogeneity was generally high. Further analyses, including subgroup analyses for different populations are needed in order to provide a more detailed and nuanced portrait of the effectiveness of digital mental health interventions. Furthermore, sensitivity analyses, notably by considering the risk of bias related to the lack of blinding of participants, would be required to minimize the risk of an overestimation of the effect. Finally, we cannot rule out the possibility of publication bias, as well as other factors that could lessen the level of confidence in the reported effects.
Available evidence suggests that digital health interventions such as internet-based cognitive behavioural therapy (iCBT) could be effective and provide an alternative to face-to-face psychological interventions to manage mental health issues in adults living with a concomitant chronic condition. In the context of the COVID-19 pandemic, digital technologies have played a key role in healthcare. Many of these innovations support the care of people in need of medical attention, including those with chronic illnesses. In response to the current crisis, but also to better prepare for the post-crisis and future crises, digital mental health interventions could be a useful tool to manage mental health problems in people living with chronic conditions.
CONCLUSION
This knowledge synthesis provides an overview of the current evidence regarding the use of digital health interventions to improve mental health in people living with a chronic condition. Knowledge users’ most urgent need was for evidence on which type of digital interventions to use for mental health management. While our meta-analysis indicates different levels of effectiveness associated with digital interventions’ characteristics, all technologies and levels of support can be used with consideration of implementation context and population.
DECLARATIONS
Ethics approval and consent to participate
Not applicable
Consent for publication
Not applicable
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Competing interests
The authors declare that they have no competing interests
Funding
This review was funded by the Canadian Institute for Health Research in the Operating Grant: Knowledge Synthesis: COVID-19 in Mental Health & Substance Use. Grant number: 202005CMS-442711-CMV-CFBA-111141
Authors’ contributions
Maxime Sasseville, Annie LeBlanc, Marie-Pierre Gagnon, Maud-Christine Chouinard, Marianne Beaulieu, Nicolas Beaudet, Pascale Cholette Christine Aspiros, Alain Larouche, Guylaine Chabot identified the need for this study and contributed to its conception and design. Maxime Sasseville, Marie-Pierre Gagnon, Mylène Boucher, Michèle Dugas, Mbemba Gisèle, Jack Tchuente and Romina Barony conducted the data collection. Jack Tchuente and Maxime Sasseville performed the data analysis. Maxime Sasseville developed the first draft of the report. All authors contributed to writing and editing and gave the final approval of the version submitted.
Study Flow Diagram: Meta-analysis Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097
Additional file 1 : Forest plot: Subgroup analysis of self-directed interventions for any digital intervention vs. usual care or another digital intervention to manage anxiety in people with any concomitant chronic condition.
Additional file 2: Forest plot: Subgroup analysis on level of professional support for any digital intervention vs. usual care or another digital intervention to manage depression in people with any concomitant chronic condition.
Additional file 3. Forest plot: Subgroup analysis on type of technology for any digital intervention vs. usual care or another digital intervention to manage anxiety in people with any concomitant chronic condition.
Additional file 4. Forest plot: Subgroup analysis on type of technology for any digital intervention vs. usual care or another digital intervention to manage depression in people with any concomitant chronic condition.
Acknowledgements
The authors would like to thank the knowledge users for their support throughout this review and the SPOR Evidence Alliance for their support.
Footnotes
maxime1_sasseville{at}uqac.ca
annie.leblanc{at}fmed.ulaval.ca
jack.tchuente.ciussscn{at}ssss.gouv.qc.ca
mylene.boucher.1{at}ulaval.ca
michele.dugas.1{at}ulaval.ca
gisele-irene-claudine.mbemba.1{at}ulaval.ca
romina.barony.1{at}ulaval.ca
maud.christine.chouinard{at}umontreal.ca
marianne.beaulieu{at}fsi.ulaval.ca
nbeaudet{at}omnimed.com
bskidmore{at}rogers.com
pascale.cholette{at}gmail.com
christine.aspiros1{at}gmail.com
alarouche{at}groupesanteconcerto.com
gchabot{at}groupesanteconcerto.com
marie-pierre.gagnon{at}fsi.ulaval.ca
REFERENCES
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81.
- 82.
- 83.
- 84.
- 85.
- 86.
- 87.
- 88.
- 89.
- 90.
- 91.
- 92.
- 93.
- 94.
- 95.
- 96.
- 97.
- 98.
- 99.
- 100.
- 101.
- 102.
- 103.
- 104.
- 105.
- 106.
- 107.
- 108.
- 109.
- 110.
- 111.
- 112.
- 113.
- 114.