The MobiliseMe study: A randomised controlled efficacy trial of a cognitive behavioural therapy smartphone application (ClearlyMe®) for reducing depressive symptoms in adolescents

1.1. Objectives of the current trial

This trial evaluated the efficacy of a CBT-based smartphone app (ClearlyMe^®) for reducing depressive symptoms in adolescents by comparing self-directed with guided use and an attention-matched control. The primary hypothesis was that relative to the control, participants who received ClearlyMe® (self-directed or guided) would have greater reductions in self-reported depressive symptoms between baseline and 6-weeks post baseline (primary endpoint), and from baseline to 4-month follow-up (secondary endpoint). Clinical importance was assessed by examining changes in clinical caseness of MDD across the conditions. This trial also examined the secondary effects of ClearlyMe^® on anxiety symptoms, psychological distress, emotional well-being, quality of life, rumination, emotion regulation, and CBT skill acquisition. It was hypothesized that relative to the control condition, participants who received the ClearlyMe^® app (self-directed or guided) would report greater improvements in the secondary outcomes at 6-weeks post baseline, and at 4- month follow-up. Finally, this trial examined participants’ engagement with ClearlyMe^®. It was hypothesised that participants who received guided support would have higher levels of engagement with the app than those who did not. The trial also included analyses to explore whether the efficacy of ClearlyMe^® was moderated by baseline depression status, gender, and openness to smartphone delivered therapies. Together, the findings will establish the efficacy, safety, and potential utility of ClearlyMe^® for disseminating first-line psychological treatment to English-speaking adolescents without clinician support.

2.1 Design

A three-arm, parallel-group randomized controlled trial (RCT) with an equal allocation ratio was conducted. The reporting follows the CONSORT checklist for RCTs ³¹ and the CONSORT extension for E-Health trials ³². Outcomes were assessed at baseline, post intervention (primary endpoint: 6-weeks post baseline), and follow-up (secondary endpoint: 4-months post baseline). The sponsor for the trial was University of New South Wales (UNSW) Sydney. Ethical approval was obtained from UNSW Human Research Ethics Committee (HC#210889). The trial protocol was prepared using SPIRIT guidelines and published prior to recruitment ³³. The trial was prospectively registered with the Australian New Zealand Clinical Trials Registry (ACTRN12622000131752) and allocated a UTN U1111-1271-8519. All participants reached the primary endpoint by May 2023, and the secondary endpoint by July 2023.

2.2 Participants

Eligible participants were Australian-residing adolescents aged between 12 to 17 years old who had access to a smartphone and the Internet; an active email address and mobile phone number; could read English at Grade 7-8 level; consent from a parent or guardian; and self-reported mild to moderately-severe depressive symptoms at screening (i.e., total score on the Patient Health Questionnaire-9 for Adolescents [PHQ-A] ranged from 5 to 19).

Participants who were receiving or planning to initiate psychological treatment for symptoms of depression or anxiety by a mental health professional; taking or planning to start prescribed medication for symptoms of depression or anxiety; reported current severe suicidal ideation (scoring ≥2 on item 9 of the PHQ-A at screening) or responded ‘yes’ to questions about serious thoughts or intentions of self-harm or suicide attempts within the past month were excluded.

2.3 Sample Size

The required sample size was 489 based on α=0.05, power=0.8, small to medium within-group effect sizes for the conditions (psychoeducation d=0.10)³⁴, self-directed CBT (d=0.35) ³⁵, and guided CBT (d=0.65) ^{10 11}, and an attrition rate of 20% between baseline and primary endpoint.

2.4 Randomisation and blinding

Randomisation adhered to the International Council for Harmonisation guidelines ³⁶. Participants were randomised after baseline to one of three trial conditions using an automated computerized randomization procedure within the Black Dog Institute Research Engine. A stratified randomization approach (1:1 ratio) with a block size of 6 ensured balance across age (12 to 14 years vs. 15 to 17 years) and symptom severity (mild [total score of ≤ 9 on the PHQ-A] vs. moderate to moderately-severe [total score of ≥10 on the PHQ-A]).

Participants who withdrew from the trial or could not engage in the intervention were discontinued. While participants were not directly informed of their allocation, they may have deduced it from differences in the interventions. The statistician was blinded to participants’ allocations for the primary outcome analyses and was unblinded for the intervention completion rates and exploratory moderator analyses. Staff who provided the guided support or trial safety procedures were unblinded to participant allocations but were not involved in data analysis.

2.5 Recruitment

Recruitment was active between May 2022 and March 2023 and consisted of paid adverts on social media (85% of participants recruited via Facebook, Instagram, TikTok and Snapchat), Google, posts on the Black Dog Institute’s website (6% of participants), word of mouth (12%), email flyers to schools, other Australian mental health organisations and Black Dog Institute mailing lists (0%).

2.6 Procedure

The trial was completed entirely online. Interested adolescents were directed to the study webpage to complete the eligibility screener. Eligible participants then completed the Adolescent Participant Information and Consent Form (A-PICF). Ineligible participants were provided with information on mental health services. Upon the provision of consent, adolescents were asked to register to the Black Dog Institute Research Engine by providing their full name, email, mobile phone number, date of birth, and account password.

Adolescents were then provided with the Parent Participant Information and Consent Form (P-PICF), which required their parents’ contact details and electronic signature. The P-PICF was required to be completed within one week and three reminders were sent. When an invalid parent email address was provided, researchers contacted the adolescent to confirm the parent’s details. Consented adolescents were to complete the baseline assessment within 7 days. Those who failed to provide parent consent or complete baseline within this time were withdrawn from the study and not randomised. Once baseline was completed, participants were randomized to one of three study conditions and received instructions via email and Short Message Service (SMS). At the primary and secondary endpoints, participants were invited via email and SMS to complete the study assessments. Each assessment took approximately 20 minutes, was available for 7 days, and could be completed on any Internet device. Participants received two reminders and were reimbursed 10AUD for each assessment completed (maximum total 30AUD). Participants in the guided condition received an additional 25AUD upon randomisation to the SMS costs. All participants were instructed to use the Get Help section in the app if additional mental health support was required during the trial, and this was not monitored by staff.

2.7 Conditions

Guided support intervention condition

Participants in this condition were provided with the same instructions and expectations for use and app access as the self-directed condition. However, these participants also received a weekly guided support chat session delivered by SMS. All SMS chats were delivered by non-clinical research staff using a standard script and decision flow chart. The SMS chats were designed to encourage participants to comply with the instructions for use and complete one collection per week. As such, the chats focused on technical support with low-intensity motivational coaching to overcome potential barriers to use. The chat sessions were designed to be completed in one sitting, lasting approximately 20 minutes, and to cease according to the chat script and decision flow chart. Chat sessions occurred between 3:30pm and 7pm Mondays to Thursday AEST. Staff initiated one additional chat attempt within two days if there was no response by participants to the initial SMS. The weekly chats were initiated regardless of prior participant responsiveness. All sessions and adherence to the script were recorded and had oversight from experienced clinical psychologists.

Control condition

This study utilised an active attention-matched control condition consisting of 6 digital psychoeducation PDF flyers that were delivered weekly via SMS. The flyer content was adapted from information used in past adolescent mental health trials ³⁴ and focused on common mental health problems, self-care activities, and included links to Australian mental health organisations. No additional contact from the research team was provided unless technical support was requested. Participants in the control condition were granted six weeks of access to ClearlyMe^® after completion of the 4-month follow-up.

2.8 Outcomes

Background characteristics

Date of birth, gender (female, male, non-binary, different identity, free response option), Aboriginal or Torres Strait Islander identity (yes, no, prefer not to say) and/or Lesbian, Gay, Bisexual, Transgender, Queer and/or Intersex identity (yes, no, I’d prefer not to say), home state or territory, categorical descriptor of home location (metropolitan, regional, or rural/remote), and school grade. Participants were asked whether they had (i) been diagnosed with depression and/or anxiety by a health professional, (ii) received psychological treatment for a mental health problem or mental illness from a health professional, (iii) taken prescribed medication for a mental health problem or mental illness like depression or anxiety, (iv) felt like they needed help for a mental health issue like depression or anxiety. Participants were asked to report whether they have ever used any mobile app to help with their emotional wellbeing or mental health (yes, no, I’d rather not say), and if yes, whether they found it helpful (yes, no). Participants were asked to rate their openness to smartphone applications for emotional wellbeing using a 5-point scale ranging from not at all (1) to extremely (5).

2.9 Data collection and analysis

Trial data was collected on the Black Dog Institute Research Engine. The final sample for analysis included all participants regardless of whether they accessed their allocated intervention. All statistical analyses were conducted in Stata. The primary analysis was by intention-to-treat. For the primary outcome, a mixed model repeated measures ANOVA was conducted to account for missing data and to include all available data for participants analysed in the trial. This approach was used to yield unbiased estimates of intervention effects in the assumption that data were missing at random. The continuous secondary outcomes were examined using the same approach. To determine changes in likely cases of major depressive disorder, baseline PHQ-A scores were dichotomised (i.e., PHQ-A scores <15 vs. ≥15) to reflect probable cases. Mixed logistic regression with a random participant intercept was used to model these binary responses. To explore the potential effects of high baseline depressive scores, an exploratory analysis was conducted with the sample restricted to only those with baseline scores indicative of probable MDD cases. To explore the effects of participants’ openness towards app therapy on primary outcome, the openness item was dichotomised as "high levels of openness to mobile apps for mental health" (scores ≥3) versus "low levels of openness" (scores ≤2). A series of models explored the relationship between condition and number of collections completed. A series of models of increasing complexity explored the relationship between condition (self-directed vs. guided) on the number of activities completed. A zero-inflated negative binomial model with a constant inflation component (i.e., not a function of condition) fitted the data better than Poisson and negative binomial models without zero inflation. Allowing inflation to differ between conditions did not significantly improve model fit. The Supplementary Material 2 outlines the exploratory analyses examining the relationship between app engagement and the primary outcome. The harms analysis was conducted using descriptive statistics and logistic regressions with relative risk ratios.

2.10 Role of funding source

The ClearlyMe® app was developed and is owned by the Black Dog Institute. The MobiliseMe trial was supported by philanthropic funding from the Goodman Foundation and chief investigators were supported by National Health and Medical Research Council (NHMRC) Grants (BOD, grant number MRF1197249, AWS, grant number GNT1197074).

3.1 Sample

Shown in Figure 1, 665 adolescents completed baseline and were allocated. A total of 569 were retained in the final baseline sample. Reasons for removal included fraudulent or duplicate responses, technical errors, or active withdrawals (see Figure 1). A total of 75.6% of the baseline sample (n=430/569) were retained at post intervention.

• Download figure
• Open in new tab

Figure 1. CONSORT diagram showing the flow of participants in the MobiliseMe trial

In the final sample, the majority were female (n=422/569; 74.2%), aged 15-17 years (n=477/569; 83.8%) and currently attending secondary school (n=536/569; 94.2%). Table 1 outlines the participants’ characteristics at baseline.

3.2 Changes in depressive symptoms (primary outcome)

Shown in Table 2 and Figure 2, the depressive symptoms of participants in the self- directed and guided conditions declined from their baseline means by a significantly greater amount than the control condition post intervention, with small effects (d=0.33 to 0.35).

• Download figure
• Open in new tab

Figure 2. Estimated marginal means for depressive symptoms (PHQ-A, primary outcome) for the conditions on each occasion of measurement.

These differential declines were not significant at follow-up.

When examining the effects in those who were likely cases of MDD at baseline, the pattern of outcomes was generally the same as the whole sample. However, the significant effects were more robust and effect sizes substantially larger post intervention (d=0.65 for guided use vs control, d=0.71 for self-directed vs control).

The number of likely cases of MDD was relatively stable in the control condition post intervention and the decline from baseline was not statistically significant (see Figure 3). The reduction in MDD cases post intervention for the self-directed condition was substantially larger than for the guided condition. When compared to the control condition, the relative odds of change in cases post intervention were statistically significant for the self-directed condition but not the guided condition (see Table 2). This was, in part, attributable to the slightly higher baseline case rate in the self-directed condition. Likely caseness rates converged by follow-up.

• Download figure
• Open in new tab

Figure 3. Predicted proportions of likely cases of Major Depressive Disorder (PHQ-A score ≥15) for the conditions on each occasion of measurement.

The exploratory subgroup analysis for gender effects (males versus females) on the primary outcome yielded similar patterns as the whole sample. This is likely due to nearly three quarters of the sample (74.2%) identifying as female with only 99 participants (17.4%) identifying as male. However, change over time in depressive symptoms post intervention was not significantly different between conditions for males alone, and the pattern of outcomes for males was different to the whole sample. The mean PHQ-A scores for females and the total sample diverged post intervention and converged at follow-up, however, the reverse occurred for males. For males, depressive symptoms in the control condition remained stable whereas symptoms in the guided and self-directed conditions declined. In cross-sectional comparisons, this difference was nearly significant at follow-up (P=.053) with a moderate effect size of d=0.58 (95%CI: -0.001 – 1.16) for guided vs control and was significant (P=.026) for self-directed vs control with a large effect size (d=0.70, 95%CI: 0.09 – 1.31).

Shown in Figure 4, changes in depressive symptoms from baseline to post intervention was significant in both the self-directed and guided conditions in participants with high levels of openness to receiving smartphone therapy (P=.009, d=0.34, 95%CI: 0.06 – 0.63 and P=.013. d=0.39, 95%CI: 0.11 – 0.66 respectively). However, the effect remained significant at follow-up in the guided condition only (P= .015, d=0.42, 95%CI: 0.12 – 0.71).

• Download figure
• Open in new tab

Figure 4. Reductions in depressive symptoms (PHQ-A) among participants with high levels of openness to smartphone therapy at baseline for the conditions at each occasion of measurement.

In participants with low levels of openness to smartphone therapy, there were no significant differences in depressive symptom change at any time point in the self-directed or guided conditions when compared to the control condition (all P >.05, see Figure 5).

• Download figure
• Open in new tab

Figure 5. Reductions in depressive symptoms (PHQ-A) among participants with low levels of openness to smartphone therapy at baseline for the conditions at each occasion of measurement.

Depressive symptoms in the self-directed condition declined at follow-up in comparison to means in the guided condition, which converged to baseline (P=.051).

3.3 Secondary outcomes

Shown in Table 3, changes in the secondary outcomes post intervention were comparable to the primary outcome, although some of the differences were smaller and less robustly significant. For psychological distress, emotional wellbeing and quality of life, the patterns of effects between conditions and over time were similar to the primary outcome. However, anxiety symptoms decreased over time in all conditions with only modest and non- significant differential change. This may be due to differences in response in participants with higher baseline anxiety scores compared to those with lower scores. There were no significant differences at follow-up for any of the outcome measures.

3.4 Engagement with ClearlyMe^® and digital working alliance in the intervention conditions post intervention

There was no significant difference in the uptake of ClearlyMe^® between the self- directed (88.7%, n=173/195) and guided conditions (90.8%, n=167/184, OR: 1.02, 95% CI:0.96 – 1.10, x² = 0.43, df=1, P=.513). Participants in the self-directed and guided conditions completed an average of 2.09 (SD: 2.82, Mdn: 1, IQR: 3) and 2.80 (SD: 2.84, Mdn: 2, IQR: 5) collections, respectively. A substantial percentage of participants in each group completed no collections. Condition was a significant predictor of completing no collections (OR: 2.93, P=.009, 95% CI: 1.31 – 6.55) but was not associated with the total number completed (IRR: 1.03, P=.818, 95% CI: .80 – 1.32). Significantly more participants undertook at least one collection in the guided condition (72.5%) compared to the self- directed condition (55.8%, relative likelihood: 1.31, P =.001, 95% CI:1.11 – 1.54). However, those in the guided condition (M: 3.86, SD: 2.66) did not go onto complete significantly more collections than those in the self-directed condition (M: 3.77, SD: 2.83, IRR: 1.02, P=.807, 95% CI: 0.85 – 1.23). Shown in the Supplementary Material 2, there was mixed evidence of a significant difference in the number of activities completed between conditions with one analysis suggesting guided (M: 13.31, SD: 9.43, Mdn: 12, IQR: 15) was superior to self- directed (M: 10.80, SD: 10.08, Mdn: 8, IQR: 15) for increasing activities completed. When examining the effects of app engagement on primary outcomes among those who completed the post intervention survey, the results suggested a degree of expectancy effect as demonstrated by the emerging symptom improvements in participants who completed no activities and the small dose effect in both conditions. The guided condition reported significantly higher total digital working alliance scores than the self-directed condition (see Table 4, t(249)=-3.14, P=.002).

3.5 Program satisfaction and barriers to use

Shown in Table 5, high levels of satisfaction were reported for all three conditions, indicating that both ClearlyMe^® and the flyers were easy to use, understand, interesting and enjoyable. The helpfulness scores for all three conditions were also similar (self-directed condition M: 3.61, SD: 0.76, guided condition M: 3.89, SD: 0.67, control conditions M: 3.64, SD: 0.83). The most common barrier to use across the three conditions was forgetfulness.

Lack of time and low motivation were also commonly reported by participants who received ClearlyMe^® but less so in the control condition. Generally, participants in the SMS-supported condition reported fewer barriers to use than participants in the self-directed condition.

Technical and user experience barriers were the least commonly endorsed barriers (<5%) across all three conditions. The qualitative analysis of free response items revealed that participants found ClearlyMe^® helpful for learning strategies to understand and change their thoughts, feelings and behaviours, as well as feel supported and connected in an easily accessible and safe space. Suggested improvements to the app included greater personalisation, more inclusive content, improved navigation, and additional features such as reminders and progress trackers. See Supplementary Material 2 for more detail.

3.5 Harms analysis

A total of 102 (17.9%) participants reported one or more expected adverse events (total of 171 events) throughout the trial. The relative risk of reporting an expected adverse event post baseline was almost double in the control condition when compared to self- directed and guided conditions with IRR of 1.73 (95%CI: 1.15 – 2.62, P=.009) and 1.98 (95%CI: 1.27 – 3.08, P=.002), respectively. Most of the expected adverse events (93 events, 54% of total events) were the presence of suicidal ideation with 77 participants reporting scores ≥ 2 on item 9 of the PHQ-A post baseline (13.5%). The relative risk of reporting suicidality was nearly double in the control condition (20.5%) when compared to self- directed (8.7%) and guided (11.4%) conditions with IRR of 2.35 (95%CI: 1.38 – 4.02, P=.002) and 1.80 (95%CI: 1.10 – 2.94, P=.019), respectively. Mental health hospitalisations accounted for 18 events (17.6%). Throughout the trial, there was 13 requests for a clinician call back but only 6 were taken by participants. Only one participant who received ClearlyMe^® reported that they perceived the app activities had made their mental health worse.

Limitations

The informativeness of this trial may be limited by the exclusion of depressed adolescents with severe suicidal ideation or recent suicidal behaviours. Future trials should endeavour to test the effectiveness of ClearlyMe^® among depressed youth with higher levels of suicidality, given that our results indicated low risk of harms from the intervention and trial protocol. Furthermore, 2 out of 5 adolescents were deemed ineligible to participate due to elevated suicidal ideation. This demonstrates the significant need and demand for digital interventions in this subgroup. Continued exclusion of suicidality in these trials will further exacerbate inequities in health access and potentially lead to the release of products that have not been tested in relevant ‘real world’ use cases⁴⁶. Future trials should also endeavour to recruit more adolescent males, those not in school, and younger adolescents, given that the low numbers of participation in these subgroups limit the generalisability of our results. This trial was strengthened by its inclusion of gender and/or sexuality diverse youth. The diversity rates are similar to those in other mental health trials in adolescents in Australia ³⁹ but are somewhat higher than the general population of adolescents ⁴⁷. This may reflect the increased need for mental health treatment and support among gender and sexuality diverse adolescents ⁴⁷ and the allyship of the Black Dog Institute for these adolescents in Australia. Lastly, the lack of differential effects for anxiety symptoms warrants further attention given that it contrasts to the findings of prior trials of dCBT ¹⁰. Future trials may benefit from adopting co-primary outcomes to ensure that trials are adequately designed and powered to detect smaller effects on anxiety symptoms, given its co-morbidity with depression in adolescents.

Conclusion

ClearlyMe^®, whether used independently or with SMS-guided support, was efficacious for reducing depressive symptoms in adolescents. The results indicated that this approach has promise for improving short-term outcomes in adolescents with mild to moderate depressive symptoms, significantly surpassing the effects of SMS-delivered psychoeducation. Future research should examine the utility of ClearlyMe^® for reducing depressive symptoms in subgroups of depressed adolescents that were under-represented in this trial, including those with suicidality, adolescent males, and adolescents not in schooling. Future translational efforts should account for adolescents’ attitudes towards mental health apps given that the effects of ClearlyMe^® appeared to be moderated by one’s openness to receiving care via this modality. This will ensure that ClearlyMe^® is being used by youth who are most likely to benefit.