Abstract
Three to seven percent of preschoolers have developmental problems or child psychiatric disorders. Randomized controlled trials (RCTs) indicate that interventions in early childhood education and care improve long-term outcomes of children from disadvantaged backgrounds. It is unknown if effects generalize beyond the well-structured context of RCTs and to children who may not have a disadvantaged background but have developmental problems or psychiatric disorders. We use data from the population-based Norwegian Mother, Father and Child Cohort Study, recruiting pregnant women from 1999 to 2008, with child follow-up from ages 6, 18, and 36 months to ages 5, 7, and 8 years. This sub-study included 2499 children with developmental problems or psychiatric disorders at age five. We investigate the effects of special educational assistance at age five on mother-reported internalizing, externalizing, and communication problems at age eight. We analyze bias due to treatment by indication with directed acyclic graphs, adjust for treatment predictors to reduce bias, and estimate effects in different patient groups and outcome domains with a hierarchical Bayesian model. In the adjusted analysis, preschoolers with special educational assistance had on average by 0.1 (0.03-0.16) standardized mean deviation weaker psycho-social difficulties in elementary school. Mean effect sizes varied between groups and outcomes. We estimate positive effects of educational assistance during the transition from preschool to the school years. It should therefore be considered as an intervention for preschoolers with developmental or behavior problems. More research with improved measurements of treatment and outcomes is needed to identify success factors for their implementation.
Between three and seven percent of preschoolers have developmental problems or child psychiatric disorders [1, 2], which are an important risk factor for mental disorders in adulthood [3]. Efforts to promote healthy growth and development in children who struggle in the early years can accordingly improve children’s long-term life opportunities [4]. This effect seems to decrease as children grow older. Therefore, investing resources later, at the age of school entry or beyond, may show less of an effect [5].
Interventions in early childhood are often described as an effective method to improve the long-term outcomes of children from disadvantaged backgrounds [6] or those with specific developmental or behavioral problems like attention deficit hyperactivity disorder, autism, or behavior or language problems [7]. Interventions in early childhood education and care (ECEC) can be especially effective because in contrast to parental training programs, their implementation relies less on parents’ abilities or motivation, and on average 93% of three to five year old children in Organisation for Economic Co-operation and Development (OECD) countries are enrolled in ECEC [8]. Randomized controlled trials (RCTs) reported clear effects of early interventions in ECEC for a horizon of up to nine months, for instance for language problems [9], children with ADHD or autism [10], and for teacher classroom management programs [11].
However, the effect sizes of such interventions are not generally large, and less is known about their effect when interventions are provided outside the well-structured context of RCTs. Even though RCTs are, due to their interval validity, the gold standard for estimating treatment effects, differences between study sample and target population and differences in treatment-implementation between study and regular care contexts, make a generalization of findings from RCT samples to populations of interest difficult [12–15]. Since RCTs often take place in a controlled setting, it may be difficult to replicate the results in other, less rigid settings. For instance, field professionals in ECEC institutions will draw on a much wider range of sources than formal experimental evidence in order inform their actions. Thus, while evidence from RCTs is encouraging, it remains unclear how it generalizes to interventions in ECEC provided in regular care.
Only a handful of studies examined the effects of special educational assistance (SEA) interventions in ECEC when they are implemented outside of RCTs. These studies used propensity scores to deal with the problematic internal validity in observational studies—due to treatment by indication—and found that children who received SEA in ECEC showed the same or worse outcomes compared children who did not receive SEA [16, 17].
The Norwegian ECEC-system facilitates the investigation of SEA, because children who cannot fully benefit from standard education and care have the right to receive free SEA. Similar to other OECD countries [2], around 4.5% of preschoolers in Norwegian ECEC have impaired functioning, the most common impairment being language and communication difficulties, followed by psycho-social difficulties and behavior problems [18]. Around 2.6% of preschoolers receive SEA, which is provided for several hours per week for individual children [18]. After stimulation of language development, social- and behavior-training are the most frequent types of SEA provided. To date, no study has–to the best of our knowledge–examined the effect of SEA in ECEC on children’s psycho-social difficulties. Related studies on SEA in Norwegian schools report that students who received it have similar or slightly worse scholastic outcomes compared to those who did not receive it [19, 20, see also 21].
In sum, the few studies examining effects of SEA in ECEC outside the context of RCTs reported small negative, to no effects of SEA. Moreover, most studies focused on educational outcomes, such that the effect of SEA on the development of psycho-social difficulties remains largely unclear. Hence, this large-scale prospective cohort study adds to the existing literature by investigating how SEA in ECEC provided outside RCTs affects the psycho-social development of children with developmental or behavioral problems.
Methods
Participants
The sample is a sub-sample of the Norwegian Mother, Father and Child Cohort Study (MoBa), a prospective population-based pregnancy cohort study conducted by the Norwegian Institute of Public Health [22, 23]. Participating mothers from all over Norway were recruited during routine ultrasound assessment in week 17 or 18 of their pregnancy in the period from 1999 to 2009. 41% of the invited women consented to participate. MoBa participants received questionnaires in gestational week 17 or 18, week 22 and week 30, at child’s age 6 and 18 months, 3, 5, and 8 years and onward. The study is still ongoing. The reported analyses also use information from the Medical Birth Registry of Norway [24]. Figure 1 shows the inclusion-flowchart.
The study sample is comprised of children whose mothers indicated developmental or behavior problems in MoBa’s age five years questionnaire, and for whom information about outcomes in the age eight years questionnaire are available. This study focuses on children with one or more of the following developmental or behavioral problems: Attention deficit hyperactivity disorder, language development, oppositional defiant or conduct disorder, autism spectrum disorder, and learning disabilities.
Materials
The current study used rating scales from MoBa questionnaires sent out at child ages five, and eight years. Exposure and inclusion criteria were based on responses in the five year questionnaire, whereas outcome measures were taken from the eight year questionnaire. The first, 1.5 and three year MoBa questionnaires and the Medical Birth Registry of Norway provided covariates.
Exposure
To measure the provision of SEA, we relied on following question: “Does your child receive, or has received any extra resources in the kindergarten?” If mothers responded “Yes” to this question, they were additionally asked about the number of hours per week. SEA is provided to individual children, both inside and outside the context of regular preschool activities.
Outcome variables
Outcome variables (PSD8 in Figure 2) were sum scores from different scales about psycho-social difficulties. Outcome dimensions were attentional, hyperactivity/impulsivity, and behavioral (ODD or CD) problems measured with the Parent Rating Scale for Disruptive Behavior Disorders (RS-DBD, [25]), emotional problems measured with the Short Mood and Feelings Questionnaire (SMFQ, [26]) and the Screen for Child Anxiety Related Disorders (SCARED, [27]), and communication problems measured with the Children’s Communication Checklist-2 (CCC-2, [28]).
Adjustment variables
Adjustment variables and those to control for loss to follow up were chosen based on the directed acyclic graph (DAG) shown in figure Figure 2. One important set of confounders includes children’s psycho-social difficulties at baseline, because these can be seen as causes of treatment and are related to later psycho-social difficulties. A number of scales in MoBa assessed psycho-social difficulties at age five and served as baseline measures (PSD5 in Figure 2). These included the Conners’ Parent Rating Scale-Revised, Short Form (CPRS-R (S), [29]), Child Behavior Checklist (CBCL, [30]), the Ages and Stages Questionnaire (ASQ, [31]), and the Children’s Communication Checklist-2.
Additional variables used for adjustment or prediction of loss to follow-up included maternal age, education, and mental health (ADHD symptoms measured with the Adult ADHD Self-Report Scale [32] at child age three and depressive symptoms measured with the SCL-5 [33] at child age five, parity, preterm birth, birth-month, hours special education per week, number of developmental of behavior problems, and contact with rehabilitation services, Child and Adolescent Psychiatric Units, or Educational and Psychological Counseling Service at child age five years.
Classification into groups with different developmental or behavioral problems
To classify if and in which area a child had developmental or behavioral problem (DBP), we used MoBa questions about mental health problems at age five. Mothers were asked if their child “suffered, or is currently suffering from any of the following long-term illnesses or health problems.” In addition, mothers were asked if they had been in contact with a Child and Adolescent Psychiatric Unit or the Educational Psychology Counseling Services and if the health problem was confirmed by a professional. Only children for whom mothers reported a health problem and who indicated that the problem was evaluated by a mental health professional were included in the sample.
Disorders or health problems for which MoBa’s age 5 questionnaire has questions included Epilepsy, Cerebral Palsy, impaired hearing, which were excluded from the current analysis, together with children for whom mothers indicated a chromosomal defect. MoBa also asked mothers about autism spectrum disorders (ASD), hyperactivity and attention problems (ADHD), language difficulties (Lang), and behavioral problems (Beh). Additional questions about learning disabilities (LD) were also used to identify cases of interest for this study. Each child was classified in one of the following DBP groups: 1. ASD, 2. LD, 3. ADHD & Beh & Lang, 4. ADHD & Beh, 5. ADHD & Lang, 6. ADHD, 7. Lang, 8. Beh. For some children, mothers indicated multiple DBP, in which case the child was assigned to the first group it fell into. If, for example, a mother indicated ASD, ADHD, and language problems, the child was assigned to the ASD group (details in supplementary materials and Table S1). The rational underlying this classification scheme was to use existing psychiatric diagnoses, and to classify children according to their most impairing problem because these have typically more severe and persistent effects psycho-social on psycho-social development.
Data analysis
All analyses were performed using R [34]. The Bayesian hierarchical regression model was implemented with the brms package [35]. The analyes are described in more detail in the supplementary material, and analysis scripts are available at https://github.com/gbiele/SPS358.
Bias from treatment by indication and loss to follow up
Estimation of treatment effects from observational data is difficult because treatment is not assigned randomly. Instead, individuals with more psycho-social difficulties at age five, who are also more likely to have psycho-social difficulties in the future, more likely receive treatment (treatment by indication). In addition, loss to follow up makes estimation of treatment effects difficult. Therefore, we used a directed acyclic graph [DAG, 36, see Figure 2] to explicate the assumed causal structure and to determine with which approach to deal with potential biases. Given this structural model, inverse probability of continued participation weighting was needed to reduce bias from loss to follow up [37], whereas adjustment for predictors of SEA was sufficient to control bias from treatment by indication. This means that we effectively estimated the effect of SEA on the change of psycho-social difficulties from preschool to elementary school.
Estimation of the treatment effects
We used a Bayesian adjusted and weighted hierarchical ordinal regression to estimate effects of SEA [35, 38, 39]. A hierarchical regression induces partial pooling (shrinkage) of estimates, which reduces the variance of estimates [40] and controls the multiple comparison problem [41]. Importantly, when anlysing related patient groups hierarchical regression results in more accurate association estimates then independent analysis of these groups [40]. We used an ordinal regression model, because the estimation of latent, normally distributed traits that underlie the rating-scale responses facilitates the presentation of results in terms of standardized mean differences (SMD). The reported results were obtained by pooling over the independent analyses of the 50 imputed data sets [42]. Consistent with recent recommendations to focus on estimation of effect sizes instead of significance testing [43, 44] we generally report means and the 90% credible intervals.
Results
The study sample includes 2499 participants (c.f., Figure 1). Thirty-three percent of the children in the sample received SEA. Table 1 describes the study sample. Figures S4 and S5 show that children with more severe problems (e.g. ASD) were more likely to receive SEA and also received SEA from better educated personnel.
Inverse probability weights reduced the differences in mean values for covariates between participants followed up and those lost to follow up to less than 0.1 SMD (c.f Figure S1; [45]). Cumulative distribution plots showed that weighting balanced the entire distributions of covariates (Figures S7 and S8).
Effects of special educational assistance
Consistent with the structural model shown in Figure 2, the analysis without adjustment showed that SEA at age five was associated with more psycho-social difficulties at age eight (c.f. Table S3 and Figure S7). Table S4 and Figures S9 and S10, S11, and S12 show coefficients of the adjusted regression model, which indicates that after adjustment for confounders SEA was associated with less psycho-social difficulties at age eight.
Over all psycho-social outcomes and groups of developmental or behavior problems the estimated average treatment effect (ATE) was a symptom reduction by 0.10 standardized mean deviations (SMD) (Credible Interval CI: 0.04, 0.16). Figure 3 shows that the 90% credible interval is for all groups above 0. The pairwise comparisons of all groups did not show clear differences in the estimated treatment effects between groups (c.f. Table S5 and Figure S14)
Figure 3 and Table 2 also show estimated effect sizes stratified by outcomes and indicate that SEA had a positive effect on all measured psycho-social outcomes. While there were some differences in the effect size estimates for different outcomes, in particular smaller effects for anxiety and communication problems, pairwise comparisons did not show reliable differences between them (c.f. Table S6 and Figure S15). Effect size estimates did not vary substantially by the child sex (c.f. Figure S18).
Discussion
This research used observational data from a longitudinal population based cohort study to investigate the effect of special educational assistance (SEA) in ECEC on psycho-social difficulties of children with developmental or behavior problems. We found that, after adjustment for treatment indicators, mothers of children who received SEA in kindergarten reported fewer psycho-social difficulties three years later, compared to mothers whose children did not receive SEA.
While there was some variation in the extent of the positive effect of SEA between groups and different psycho-social difficulties, these differences were not reliably different from zero (c.f. Figures S14 and S15). Because the credible intervals for these differences are large compared to the magnitude of the estimated overall effect and the random effects standard deviations are clearly non-zero (S4), these results do not exclude the possibility of group differences. Instead, they might reflect difficulties in reliably measuring exposure, covariates, and outcomes based on parent reports only. Still, the available data were sufficient to reveal an overall positive effect of SEA.
While the positive effect reported in this study is consistent with the results of randomized controlled trials [10, 11] and with reports of the positive effects of preschool child care quality [46], it also stands in contrast to previous observational studies, which estimated no or a small negative “effects” of special education. This apparent contradiction can be due to a number of differences between the current and previous studies. We had estimates of pre-treatment difficulties, and could estimate effects of special education on the change of psycho-social difficulties. Moreover, we used adjustment for treatment predictors instead of propensity score weighting. Adjustment is the preferable approach if treatment-predictors are not colliders on a backdoor path from outcome to treatment and if the sample size is large enough to allow for inclusion of many of adjustment variables. Another important difference is that whereas previous studies focused on scholastic outcomes, we focused on the effect on psycho-social difficulties. This is a to date little examined but important outcome of SEA, because early psycho-social difficulties are associate with impaired functioning in adulthood [3]. Interestingly, the clear results of SEA on externalizing behavior suggests that in addition to helping children with DBP, it can also benefit their families by reducing disruptive behavior.
The estimated effect size for the reduction of psycho-social difficulties is with on average 0.10 standardized mean difference small. In comparison, previous meta analysis about school– or ECEC–based interventions found effect sizes of between -.3 and 1.3 SMD for children with or at risk for ADHD [47, 48] or SMD between 0.3 and 1.1 for children with autism [49]. Randomized trials of classroom management training for kindergarten teachers showed effect sizes similar to our results [Cohen’s d around 0.3 for high risk children at the nine-months follow up, 50]. It is possible that the small effect sizes we estimated are, in addition to above mentioned measurement problems, due to the fact the SEA was often provided by personnel with limited training, especially for children with typically less severe problems (c.f. Figure S5). More generally, the decentralized organization of the Educational and Psychological Counselling Service is likely to lead to a large variation in the implementation of SEA [51]. MoBa did not collect more detailed data about SEA, which could help to elucidate when it is most effective. Another possible explanation is that the composition of the study sample, which over-represents well-educated families compared to the population [37], leads to an underestimation of the true effect size, because well-educated parents could reduce children’s psycho-social difficulties even without SEA [52].
While the current study showed that mothers report fewer psycho-social difficulties in elementary school when their children received SEA in ECEC, a causal interpretation of this result as reflecting an effect of SEA rests on a number of assumptions encoded in Figure 2. One un-testable assumption is that there are no unmeasured confounders that predict both which children receive SEA and their developmental pathway. Even though the reported analysis includes obvious confounders, other unobserved confounder could still account for some of the positive association of SEA and psycho-social development. However, because RCTs of SEA and similar interventions typically report positive effects, and thus confirm a causal role of SEA, it appears unlikely that the effects estimated in this study are primarily due to confounding.
Given that the current study does not have more precise and detailed measures, future studies that assess outcomes through blinded raters or objective instruments and measure quality and quantity of the treatment more thoroughly are needed. Studies that better assess variations in treatment quality and content and use more representative samples will be useful to investigate reasons for the relatively small effects observed in the current study, and to identify criteria for effective interventions in ECEC.
Conclusion
Previous RCTs about special educational assistance and teacher management programs showed that interventions in ECEC have a positive immediate impact for children with developmental or behavioral problems, but provide little guidance on long-term effects. The current study has due to its observational character a lower internal validity than RCTs, but complements them in terms of external validity and by examining long-term effects. It thus strengthens the view that interventions in ECEC are a useful approach to support preschoolers with developmental or behavioral problems.
In sum, the current study suggests that the psycho-social development of children with developmental or behavior problems can be modified in a positive way through interventions in ECEC, also when provided outside the structured context of randomized controlled trials. Future research with better measurements and more representative samples should investigate under which conditions such interventions are most effective.
Key points and relevance
Parent training programs are considered a key component of early interventions against the development of developmental or mental health problems
Special educational assistances in early childhood education and care (SEA in ECEC) showed positive effects on later outcomes in RCTs, but population based cohort studies reported no or even negative associations
Results from our large, population based cohort study indicate that SEA in ECEC is associated with reduced psycho-social difficulties in elementary school
SEA in ECEC may be effective also when implemented outside the structured context of RCTs and for children who do not come from disadvantaged backgrounds
Easy access to SEA in ECEC may be a component of early intervention strategies to prevent or mitigate development of psycho-social difficulties in preschoolers at risk.
Data Availability
Data are available upon application to the Norwegian Mother, Father and Child Cohort Study (MoBa) https://www.fhi.no/en/studies/moba/
Data Availability
Data are available upon application to the Norwegian Mother, Father and Child Cohort Study (MoBa) https://www.fhi.no/en/studies/moba/
Footnotes
Author Note The authors made the following contributions. Guido Biele, PhD: Conceptualization, Formal analysis, Funding acquisition, Writing - Original Draft Preparation, Writing - Review & Editing; Ratib Lekhal, PhD: Conceptualization, Writing - Original Draft Preparation, Writing - Review & Editing; Kristin R. Overgaard, PhD: Conceptualization, Funding acquisition, Writing - Review & Editing; Mari Vaage Wang, PhD: Conceptualization, Writing - Review & Editing; Ragnhild Eek Brandlistuen, PhD: Conceptualization, Writing - Review & Editing; Svein Friis, PhD: Conceptualization, Writing - Review & Editing; Paal Zeiner, PhD: Conceptualization, Funding acquisition, Writing - Review & Editing.