Joint engagement is associated with greater development of language and sensory awareness in children with autism

Participants

Participants were users of a language therapy app that was made available gratis at all major app stores in September 2015. Once the app was downloaded, caregivers were asked to register and to provide demographic details, including the child’s diagnosis and age. Caregivers consented to anonymized data analysis and completed the Autism Treatment Evaluation Checklist (ATEC) (Rimland & Edelson, 1999), an evaluation of the receptive language using the Mental Synthesis Evaluation Checklist (MSEC) (Braverman et al., 2018), as well as the Screen Time assessment and the Diet and Supplements assessment. The first evaluation was administered approximately one month after the download. The subsequent evaluations were administered at approximately three-month intervals. To enforce regular evaluations, the app became unusable at the end of each three-month interval and parents were required to complete an evaluation to regain its functionality.

Time with an adult actively involved in the same activity

Participants were required to respond to the question: “How much time, on average, does your child spend with an adult actively involved in the same activity (talking, reading, playing, therapy) each day”? To assess the effect of joint engagement, we compared participants who indicated that they spend on average 1 hour or less per day (N=2272; the low joint engagement group, abbreviated lowJE) to participants who reported on average 3 or more hours of joint engagement (N=2578; the high joint engagement group, abbreviated highJE).

Evaluations

A caregiver-completed Autism Treatment Evaluation Checklist (ATEC) (Rimland & Edelson, 1999) and Mental Synthesis Evaluation Checklist (MSEC) (Braverman et al., 2018) were used to track child development. The ATEC questionnaire is comprised of four subscales: 1) Speech/Language/Communication, 2) Sociability, 3) Sensory/Sensory awareness, and 4) Physical/Health/Behavior. The first subscale, Speech/Language/Communication, contains 14 items and its score ranges from 0 to 28 points. The Sociability subscale contains 20 items within a score range of 0 to 40 points. The third subscale, referred here as the Sensory awareness subscale, has 18 items and score range from 0 to 36 points. The fourth subscale, referred here as the Health subscale, contains 25 items and score range from 0 to 75 points. The scores from each subscale are combined in order to calculate a Total Score, which ranges from 0 to 179 points. A lower score indicates lower severity of ASD symptoms and a higher score indicates more severe symptoms of ASD. ATEC is not a diagnostic checklist and it was designed to evaluate the effectiveness of treatment (Rimland & Edelson, 1999). Therefore, ASD severity can only have an approximate relationship with the total ATEC score and age. Table 1 lists approximate ATEC total score as related to ASD severity and age as described in Mahapatra et al. (Mahapatra, Khokhlovich, et al., 2018).

ATEC was selected as a tool since it is one of the few measures validated to evaluate treatment effectiveness. In contrast, another popular ASD assessment tool, Autism Diagnostic Observation Schedule or ADOS, (Lord et al., 2000) has only been validated as a diagnostic tool. Various studies confirmed the validity and reliability of ATEC (Geier et al., 2013; Jarusiewicz, 2002) and several trials confirmed ATEC’s ability to longitudinally measure changes in participant performance (Charman et al., 2004; Klaveness et al., 2013; Magiati et al., 2011; Mahapatra, Khokhlovich, et al., 2018). Moreover, ATEC has been used as a primary outcome measure for a randomized controlled trial of iPad-based intervention for ASD, named “Therapy Outcomes By You” or TOBY, and it was noted that ATEC possesses an “internal consistency and adequate predictive validity” (Whitehouse et al., 2017). These studies support the effectiveness of ATEC as a tool for longitudinal tracking of symptoms and assessing changes in ASD severity.

Expressive language assessment

The ATEC Speech/Language/Communication subscale includes the following questions: 1) Knows own name, 2) Responds to ‘No’ or ‘Stop’, 3) Can follow some commands, 4) Can use one word at a time (No!, Eat, Water, etc.), 5) Can use 2 words at a time (Don’t want, Go home), 6) Can use 3 words at a time (Want more milk), 7) Knows 10 or more words, 8) Can use sentences with 4 or more words, 9) Explains what he/she wants, 10) Asks meaningful questions, 11) Speech tends to be meaningful/relevant, 12) Often uses several successive sentences, 13) Carries on fairly good conversation, and 14) Has normal ability to communicate for his/her age. With the exception of the first three items, all items in the ATEC subscale 1 primarily measure expressive language. Accordingly, the ATEC subscale 1 is herein referred to as the Expressive Language subscale to distinguish it from the Receptive Language subscale tested by the MSEC evaluation.

Receptive language assessment

The MSEC evaluation was designed to be complementary to ATEC in measuring complex receptive language. Out of 20 MSEC items, those that directly assess receptive language are the following: 1) Understands simple stories that are read aloud; 2) Understands elaborate fairy tales that are read aloud (i.e., stories describing FANTASY creatures); 3) Understands some simple modifiers (i.e., green apple vs. red apple or big apple vs. small apple); 4) Understands several modifiers in a sentence (i.e., small green apple); 5) Understands size (can select the largest/smallest object out of a collection of objects); 6) Understands possessive pronouns (i.e. your apple vs. her apple); 7) Understands spatial prepositions (i.e., put the apple ON TOP of the box vs. INSIDE the box vs. BEHIND the box); 8) Understands verb tenses (i.e., I will eat an apple vs. I ate an apple); 9) Understands the change in meaning when the order of words is changed (i.e., understands the difference between ‘a cat ate a mouse’ vs. ‘a mouse ate a cat’); 10) Understands explanations about people, objects or situations beyond the immediate surroundings (e.g., “Mom is walking the dog,” “The snow has turned to water”); MSEC consists of 20 questions within a score range of 0 to 40 points; similarly to ATEC, a lower MSEC score indicates a better developed receptive language.

The psychometric quality of MSEC was tested with 3,715 parents of ASD children (Braverman et al., 2018). Internal reliability of MSEC was good (Cronbach’s alpha > 0.9). MSEC exhibited adequate test– retest reliability, good construct validity, and good known group validity as reflected by the difference in MSEC scores for children of different ASD severity levels. MSEC norms have been reported earlier (Arnold & Vyshedskiy, 2022).

To simplify interpretation of figure labels, the subscale 1 of the ATEC evaluation is herein referred to as the Expressive Language subscale and the MSEC scale is referred to as the Receptive Language subscale.

Statistical analysis

The framework for the evaluation of score changes over time has been earlier explained in minute detail (Mahapatra, Khokhlovich, et al., 2018; Vyshedskiy et al., 2020). In short, the concept of a “Visit” was developed by dividing the three-year-long observation interval into 3-month periods. All evaluations were mapped into 3-month-long bins with the first evaluation placed in the first bin. When more than one evaluation was completed within a bin, their results were averaged to calculate a single number representing this 3-month interval. Thus, we had 12 quarterly evaluations for both highJE and lowJE groups.

It was then hypothesized that there was a two-way interaction between pretend-play-group and Visit. Statistically, this hypothesis was modeled by applying the Linear Mixed Effect Model with Repeated Measures (MMRM), where a two-way interaction term was introduced to test the hypothesis. The model (Endpoint ∼ Baseline + Gender + Severity + Sleep-Problem-Group * Visit) was fit using the R Bioconductor library of statistical packages, specifically the “nlme” package. The subscale score at baseline, as well as gender and severity were used as covariates. Conceptually, the model fits a plane into n-dimensional space. This plane considers a complex variability structure across multiple visits, including baseline differences. Once such a plane is fit, the model calculates Least Squares Means (LS Means) for each subscale and group at each visit. The model also calculates LS Mean differences between the groups at each visit.

In preparation for statistical analysis, participants in the lowJE group were matched to those in the highJE group using propensity score (Schneider et al., 2007) based on age, gender, expressive language, receptive language, sociability, sensory awareness, and health at the 1^st evaluation (baseline). The number of matched participants was 2272 in each group, Table 2.

Limitations

Epidemiological studies of app users provide access to a large number of children at relatively low cost, but have obvious downsides, such as relying on parent reports for diagnosis and assessments. Participants of this study identified their diagnosis as ASD at their last evaluation. Other diagnostic options included: Suspected ASD, Mild Language Delay, Pervasive Developmental Disorder, Attention Deficit Disorder, Social Communication Disorder, Specific Language Impairment, Apraxia, Sensory Processing Disorder, Down Syndrome, Lost Diagnosis of Autism or PDD, Other Genetic Disorder, Normally Developing Child.

Diagnosis misidentification should not have been a significant problem. The app is popular only in the ASD community. Most app users (82%) reported ASD diagnosis by their last evaluation. If ASD participants were mislabeling themselves as a different condition, they would have been excluded from this study. Second, if participants without ASD diagnosis were misidentifying themselves as ASD, they must have also misrepresented their ATEC score (calculated based on 77 questions) as the average initial ATEC total score was 69.0 ± 26.2, which corresponds to moderate-to-severe ASD as reported earlier (Mahapatra, Vyshedsky, et al., 2018) and shown in Table 1. It is highly unlikely that participants went through such an effort to consciously misrepresent their condition since there was no benefit in doing so, although random events of this kind cannot be completely excluded.

We have also previously explored the relationships between different levels of ASD severity reported by parents and the reported scores. We hypothesized that if parents clearly understood and honestly communicated their child’s diagnosis, the reported ASD severity level would have a consistent relationship with assessment subscales. Wherein, greater ASD severity would correspond to worse assessment scores and vice versa. Conversely, if parents misreported their children’s diagnosis, no difference in the average assessment score would be expected between the groups. This cross-sectional analysis of 9573 children has demonstrated statistically significant differences between mild and moderate ASD diagnosis as well as between moderate and severe ASD diagnosis in each subscale and in every age group in children 3 years of age and older (Jagadeesan et al., 2022). These findings contribute to support good fidelity of parents’ reports of children’s diagnosis.

In terms of evaluation scores, two biases are possible. Parents may yield to wishful thinking and overestimate their children’s abilities on a single assessment (Scattone et al., 2011). This, however, is not a problem for a longitudinal study as the pattern of changes generated by measuring the score dynamics over multiple assessments provides meaningful data on a child’s developmental trajectory even if wishful thinking was inflating each individual assessment. A second possible bias concerns parents who may wish to exaggerate their child’s improvement by purposefully entering a score that is better than their previous assessment. If this was the case, one would surmise that parents who have invested more time and energy into the app were also more likely to rate their child as improving. We have previously addressed this possibility analytically by calculating the correlation coefficient between the app usage measured in days/week with child’s improvement in receptive language (r = –0.01), expressive language (r = –0.06), sociability (r = –0.04), cognitive awareness (r = –0.01), and health (r = 0.01) (Vyshedskiy et al., 2020). Low absolute values of correlation coefficients, as well as the variability of the direction of correlation (positive correlation for improvement of health and negative correlation for other subscales) were not consistent with the hypothesis that parents who invested more time into working with the app were also more likely to rate their child as improving. Additionally, even if parents wanted to score their children as improving (consciously or unconsciously), this would have been very difficult as parents were blinded to their answers at all the previous evaluations and it is near-impossible to recall one’s answers to 133 questions (with each question coming with 3 to 6 options) for three months (time between evaluations). We conclude that it is unlikely that evaluation biases influenced the results of this study.

Another limitation is that this study observational design cannot definitively prove causality since not all confounders can be adjusted appropriately. Our results only inform on the association of joint engagement with better language acquisition and sensory awareness. However, this report in combination with the multitude of randomized controlled studies of joint engagement interventions points to the existence of strong causality between joint engagement and a child’s developmental trajectory.