The impact of a demand-side sanitation and hygiene promotion intervention on sustained behavior change and health in Amhara, Ethiopia: a cluster-randomized trial

Ethics and trial registration

Ethical approval for the Andilaye Trial was provided by Emory University (IRB00076141), the London School of Hygiene & Tropical Medicine (9595), and locally by the Amhara Regional Health Bureau (HRTT0135909). The trial was registered with clinicaltrials.gov (NCT03075436) on March 9, 2017.

Study design

This parallel CRT was conducted in West Gojjam and South Gondar Zones of the Amhara National Regional State, a region of Ethiopia in which WASH conditions are inadequate [46], slippage in sanitation coverage and improved sanitation behaviors has been documented, and several NTDs (e.g., soil-transmitted helminths, trachoma) are hyperendemic [47]. The specific behaviors targeted by the Andilaye intervention and their respective key outcome indicators are detailed in Table 1. We sought to investigate whether any changes in WASH behaviors targeted by the Andilaye intervention were sustained, and we tracked intervention fidelity through a process evaluation.

Fifty kebeles— the smallest government administrative unit in Ethiopia—in Amhara Region were selected and randomly assigned to either the Andilaye intervention or counterfactual (i.e., control comparison group). Kebele clusters, the randomization unit, were rural or peri-urban sub-districts from three districts (woredas)— Bahir Dar Zuria, Fogera, and Farta. These areas were targeted because they represented different topographical conditions present in Amhara. Following baseline data collection, we randomly selected and assigned clusters using stratified sampling by district. Randomization was done using a computer-based random number generator. To secure balance across three key potential confounders (i.e., latrine coverage, washing station with soap coverage, and head of household education), we established a priori that the intervention and control mean values for these three variables, using baseline survey data, should be within two standard deviations of the overall mean. As such, the randomization process was repeated twice using replacement rerandomization [49] to achieve balance according to that a priori criterion. Figure 2 provides further details in a flow diagram.

“Andilaye” Intervention

The Andilay intervention motto was “Together we can be a strong, caring, healthy community”. Intervention activities offered aspirational messages that emphasized the need for collective action to make positive change in the community and used verbal persuasion to enhance collective efficacy perceptions [50]. The Andilaye intervention focused on three WASH-related behavioral themes, informed by formative research: (1) sanitation, (2) personal hygiene, and (3) household environmental sanitation. Within these themes were 11 constituent practices targeted by the intervention (Table 1); these practices were identified through formative research as ones that could be targeted using demand-side approaches, and were seen as achievable, per stakeholder feedback [42].

Intervention activities and behavior change tools were informed by our formative research and specifically designed to incorporate techniques that addressed behavioral factors such as action knowledge, perceived personal and household barriers to behavioral adoption, identification and planning, and behavioral control perceptions amongst others [42]. Activities occurred at four levels-district, community, group, and household (Supplemental Figure 1 and Supplemental Table 1). Key activities included community mobilization and commitment events, community conversations, and household counseling visits with caregivers—all of which were guided by behavior change tools (e.g., community commitment banner, community conversations flipbook, and household counseling flipbook and goal cards) with illustrations produced by an artist based in Ethiopia.

The Andilaye intervention commenced with district-level capacity building activities, such as action planning and training of trainers who would facilitate intervention activities. Further, district-level refresher trainings and adaptive management activities were conducted to reinforce previously acquired knowledge and skills, address trainer/facilitator turnover, and review successes and address challenges faced in implementing group and household level activities. Community-level activities included the ‘Whole System in the Room’ [51], community mobilization and commitment events, and cross-fertilization visits. During household-level counselling visits, trained Women’s Development Army Leaders (WDALs) provided personalized counselling to caregivers to equip them with the knowledge, skills, and motivation necessary to adopt and maintain improved WASH practices. Structured community conversations, implemented by trained community facilitators, provided further opportunity for group-level counselling and support.

• Download figure
• Open in new tab

Figure 1. The Andilaye Trial consists of three major phases:

(1) formative research and intervention design, (2) intervention implementation and process evaluation, and (3) impact evaluation. Kebele and household enrollment took place during baseline data collection (March to April 2017). Implementation of Andilaye intervention activities began in September 2017 and continued through midline data collection (March to April 2018), quarterly monitoring (June to July and November to December 2018), and endline evaluation (March to May 2019). See Supplemental Table 2 for specific dates of the delivery of intervention activities. Midline data reflected at least 2 months since the start of household-level behavior change activities and 3 weeks since the completion of a catalyzing community-level mobilization and commitment event. Our endline data reflected the implementation of 14-15 months of household-level behavior change activities and 6-7 months of group-level behavior change activities (as dictated by trained activity facilitators) and 13-14 months since the community mobilization and commitment events.

The Andilaye intervention was delivered through Ethiopia’s HEP, via trained government-salaried Woreda (district) Health Office officials, HEWs, and volunteer WDALs (Figure 1; see Supplemental Table 2 for an alignment of relevant roles and responsibilities of the HEP and Andilaye Trial). Implementation of the Andilaye intervention was overseen by an Ethiopian-based study team. The Health Extension Services Package, and its accompanying CLTSH module delivered via the HEP, were being scaled throughout Ethiopia [52] and reflected the existing government-supported demand-side sanitation and hygiene approach. No attempt was made to modify the roll out of this standard care of WASH programming in any of our study clusters. Results from our equivalence analyses indicated balance in the number of previously CLTSH-triggered and ODF certified kebeles, between study arms (Figure 2). Baseline statistics, along with the fact that 39 of 50 kebele clusters randomly selected for inclusion in the Andilaye Trial had been triggered with CLTSH, and certified ODF, provided strong evidence that behavioral slippage was, indeed, an issue that needed to be addressed in Amhara and perhaps elsewhere in Ethiopia.

• Download figure
• Open in new tab

Figure 2.

Study flow diagram

Outcomes of interest

Survey instruments administered for our impact evaluation collected data on key outcomes through self-reports from respondents and other household members. Primary outcomes of interest included mental health and three targeted WASH behavioral themes (1) sanitation, (2) personal hygiene, and (3) household environmental sanitation behaviors, consisting of 11 constituent practices (Table 1). To measures WASH outcomes, we pulled from standard WASH indicators, and leveraged formative research data to contextually adapt survey prompts and answer choices (shown in Supplemental Table 4 of Delea et al., 2019). Sustainability of WASH-related behaviors was measured through the proportion of individuals and households consistently practicing target behaviors at midline and endline.

For mental health, we assessed subjective well-being using the validated WHO’s Weil-Being Index (WHO-5) [53] and symptoms of anxiety, depression, and non-specific emotional distress using the Hopkins Symptom Checklist (HSCL) [54]. The WHO-5 asks the respondent to indicate how frequently they relate to each of five statements in the previous two weeks using a five-point Likert scale. Higher scores are better (range: 0-25) with scores below 13 indicating poor well-being. The HSCL is a non-diagnostic tool that includes 25 items to assess symptoms of anxiety (items 1-10), depression (items 11-25) and overall emotional distress (all 25 items). We omitted two items from the depression set: an item on sexual desire, which was deemed inappropriate for unmarried women, and an item on suicide ideation, because we were unable to provide clinical recourse if needed. Participants indicated how much symptoms bothered them in the previous week (‘not at all’ [1] to ‘Extremely’ [4]). The final score for each state is a mean of responses for each of the relevant items (range from 1 to 4). Scores of 1.75 or higher indicate that the condition could be present while lower scores are an indication of lower anxiety, depression, or distress.

Secondary outcomes included diarrhea period prevalence, measured through caregiver report of the index child (i.e., youngest child in the household aged one to nine years at baseline), sanitation insecurity, and water insecurity. For sanitation insecurity, we asked respondents to indicate how often (never, sometimes, often, always) they felt one of seven different forms of sanitation insecurity (i.e., 7 factors). Scores were means of all items in the factor. A higher score represents higher sanitation insecurity. The factors were predesignated, and based on a validation that was done in another study [24]. Water insecurity was measured through the Household Water Insecurity Experiences (HWISE) scale [55]. HWISE includes 12 items with four response categories (never, rarely, sometimes, often/always). The score is the sum of responses, ranging from 0-36. A higher score indicates greater household water insecurity, and we considered households to be water insecure with a score of 12 or more, as described elsewhere [55]. Measures of collective efficacy [50] and intermediate behavioral antecedents were also collected, but are not reported here.

Process evaluation

We conducted a process evaluation alongside our impact evaluation to describe and analyze key aspects of the Andilaye’s implementation and provide insights and understanding of program impacts. We defined fidelity as the degree to which the intervention or program was delivered as intended [56]. Quantitative process data on dose delivered, participation, and dose received were collected through the direct observation of all district and community-level activities—these activities being facilitated or co-facilitated by the Ethiopian-based study team. Questions were incorporated into our survey instruments administered during quarterly monitoring and endline data collection to capture exposure to key Andilaye behavior change activities by respondents from study-enrolled households in intervention kebeles. This included self-reported awareness of and attendance at the community mobilization and commitment event and community conversations, and the number of household counseling visits received from WDALs. Per protocol, all community members were targeted to attend the community mobilization and commitment event; routine (1-2 per month) community conversations primarily focused on influential community members (e.g., male heads of households, religious leaders, mother-in-laws) targeted in the ‘Whole System in the Room’; and caregivers were to receive monthly household counseling visits (each visit lasting approximately 30 minutes) following the typical structure for the WDAL and in accordance the HEP.

Sample size and power

A detailed description of sample size considerations is published in the study protocol [42]. Briefly, we powered this study on mental well-being outcomes, as measure by the HSCL [54], utilizing data from Ethiopia and East Africa suggesting that approximately 20-35% of rural women experience elevated symptoms of common mental disorders such as anxiety and depression [57,58]. Our sample size determination indicated we should recruit and enroll 25 households from each of 50 kebele study clusters, with 25 kebeles per study arm, targeting one index child—the youngest child aged one to nine years at baseline—per household. We increased our final sample size to accommodate for 20% of households being lost to follow-up. Our target sample, therefore, included 30 households in each kebele study cluster, or 1,500 households in total (i.e., 750 per study arm).

Data collection

Data were collected via structured household interviews and observations by trained enumerators during rounds of data collection. Surveys were collected using mobile phones equipped with the freely available Open Data Kit (http://opendatakit.org/). We interviewed the same households at each round of data collection. Figure 1 and Supplemental Table 2 provide a summary of the timeline of intervention implementation in relation to points of data collection for our impact evaluation.

Analytical methods

We followed a pre-analysis plan developed following baseline data collection [42]. The primary analysis method was an “intention-to-treat” analysis, which compares the intervention arm to the counterfactual arm without regard to intervention fidelity or compliance. The majority of our primary and secondary outcomes were binary variables, and for these we used log-linear binomial regression models and report the prevalence ratio (PR). For these binary outcomes we also present prevalence differences (PD), which were calculated using post-estimation commands to estimate the average marginal effects. For continuous outcomes, such as the WHO-5 and HSCL scores, we used linear regression models. All models included an intervention variable as a fixed effect, account for the stratified design through the inclusion of the woreda indicator variable [59], and incorporated generalized estimating equations with robust standard errors to account for the clustering of observations within kebeles. For each of our primary outcomes of interest, we assessed if there was interaction across various sub-groups, including exposure to previous CLTSH triggering and sex of the index child. We also assessed if water insecurity modified the effectiveness on hygiene behaviors. For all of these analyses, we included interaction terms to test if effect modification was present (i.e., the interaction term had a p-value <0.05).

To assess whether any improvements in WASH behaviors were sustained between the midterm and the follow-up period, we compared the prevalence of key targeted sanitation, hygiene, household environmental sanitation indicators between the demand-side intervention arm and the counterfactual arm group using the baseline, midline, and endline data (Figure 1). Based on the endline data, at the time of data collection household-level activities had taken place for the past 14-15 months, group-level activities for the past 6-7 months, and community-level mobilization and commitment events were completed 13-14 months prior. Elements of the intervention were still ongoing during the time of our endline data collection, and we did not collect further data after the endline visit.

Survey results

Our baseline assessment showed balance in terms of our primary outcomes of interest and demographic variables [42]. Our endline results reflect complete data from 1,472 (93%) of 1,589 households enrolled in the study at baseline, and was well within our sample size requirement of 1,250 households. Of the 793 enrolled intervention households and 796 enrolled counterfactual households, retention was similar in both arms, at 94% and 92%, respectively (Figure 2). A large majority (90%) of the respondents were female, by design. Of these 1,472 respondents, 85% were the mother of the index child.

Process evaluation

Reports from Woreda Health Offices collected at endline indicated that none of the 50 kebele study clusters (intervention or counterfactual) received additional CLTSH triggering or re-triggering during the course of the Andilaye Trial. The process evaluation survey was conducted at 703 randomly selected study-enrolled households in the months following the event (i.e., quarterly monitoring visits).

For the Andilaye intervention, the fidelity of action planning workshops and trainings at the district and community-levels were high (See Supplemental Table 3 for a summary of process data on dose delivered, participation, and dose received). In each district (woredas), we conducted a sensitization and action planning workshop, trainings of trainers and intervention activity facilitators (including one round of refresher trainings) and an adaptive management workshop. All intervention kebeles (n=25) had delivery of the ‘Whole System in the Room’ and action planning activity, community mobilization and commitment event, and skill-based training of WDALs (including two rounds of review meetings and refresher trainings).

Respondent-reported exposure of key behavior change activities varied, but was generally suboptimal (Figure 3). Overall, only 18% of respondents reported attending the community mobilization and commitment event, and 22% reported being aware of the activity, according to data collected from intervention household respondents via process evaluation surveys administered months following quarterly monitoring visits. WDALs and their supervisors (i.e., HEWs) were trained to facilitate monthly counseling visits with households in their catchment area. However, among intervention household respondents of process evaluation surveys administered at endline (n=665), only 59% reported receiving a counseling visit and 43% reported receiving at least one follow-up visit. No intervention kebeles had WDALs conducting counseling visits monthly. Among household respondents reporting at least one household visit (n=391), the average number of visits reported was 2-3 during the 14-15-month implementation period (Supplemental Table 3). Among respondents of the group-level process evaluation surveys administered at endline (n=707), 28% reported attending at least one community conversation, and 46% reported being aware of the activity.

• Download figure
• Open in new tab

Figure 3. Respondent-reported exposure of key behavior change activities of the Andilaye intervention.

Respondents from study-enrolled households in intervention kebeles (n=793) were surveyed on their awareness and attendance in the community-level mobilization and commitment event during quarterly monitoring visits and awareness and attendance of group-level community conversations and frequency household-level counseling visits received by endline visits. A total of 703 (89%) and 707 (89%) surveys with responses to process evaluation prompts were completed from quarterly monitoring and endline, respectively. Reported frequency of counseling visits reflected a total of 665 survey responses as these questions were not relevant for study-enrolled households that were residents of caregivers who were trained as Women’s Development Army Leaders (WDALs) responsible for conducting the Andilaye counselling visits (n=42).]

Impacts on sanitation, personal hygiene, and household environmental sanitation

The intervention did not increase latrine access. At endline, 62% of both intervention and control households had at least one latrine (prevalence ratio [PR] 0.99; 95% CI: 0.82, 1.21) (Table 1). There was no difference in the prevalence of improved latrines (PR 1.13; 95% CI: 0.81, 1.59) or in fully constructed latrines (PR 1.15; 95% CI: 0.86, 1.54). Although there were improvements in many latrine characteristics in the intervention arm compared to the counterfactual arm (e.g., presence of water available or cleansing agent near or inside the latrine for handwashing, and water available for flushing or self-cleansing), the conditions (e.g., presence of feces on floor) of latrines in the intervention arm were often poor (Supplemental Table 4).

The intervention did not impact defecation practices. Overall, 40% of respondents reported practicing open defecation during the previous two days; only 46% of respondents had defecated in any latrine during the previous two days (Supplemental Table 4). All measures of latrine utilization and non-utilization were similar across intervention and counterfactual arms. This includes indicators of respondent open defecation (PR: 1.05; 95% CI: 0.76, 1.45), safe disposal of child feces (PR: 0.96; 95% CI: 0.69, 1.32), and number of people from another household who used a latrine during last seven days (difference: -0.40; 95% CI: -0.85, 0.05).

The intervention did not impact personal hygiene behaviors. The prevalence of washing stations with water (PR: 0.96; 95% CI: 0.72, 1.26) was similar between the intervention and counterfactual arms. Presence of hand or face washing stations were observed in 98% of households (Table 1), although water and soap were observed in only 20% of handwashing stations and 2% of facewashing stations (Supplemental Table 4). The prevalence of stations with soap was higher in the intervention arm, although only 3% of households in this arm had a washing station with soap present. Among all children aged 1-9 years at endline, observations of facial cleanliness indicated 29% had ocular discharge, 38% had wet nasal discharge, 44% had dry nasal discharge, and 50% had dust, dirt, or debris on their faces (Table 1). There were no meaningful differences between the study arms for any of these facial cleanliness measures.

We found no evidence that the Andilaye intervention impacted household environmental sanitation. Across both arms, the majority of respondents and heads of household had animal herding responsibilities (88% overall), and animal feces were present in the compound in 82% household compounds (Supplemental Table 4). A similar proportion of households in intervention and control kept animals separate from the house (PR=1.01, 95% CI 0.91, 1.11). About half of households did not leave animal feces/waste in the open (Table 1); this was similar between the intervention and counterfactual arms (PR=1.10; 95% CI: 0.95,1.28).

Sustained changes of key indicators

There was little difference in the sustainability of key targeted indicators — assessed by changes between midline and endline — on sanitation access and practices, personal hygiene access and practices, and household environmental sanitation over the course of follow up (Figure 4). At midline, most variables continued to show little difference between the intervention and counterfactual arms, although the prevalence of drop hole covers in latrines and the prevalence of appropriate hygiene behaviors were more common in the intervention arm. At the endline visit, the prevalence of drop hole cover was largely sustained in the intervention arm, while the prevalence of drop hole covers decreased in the control arm (PR=1.77’ 95% CI: 1.19, 2.63). All other variables at endline had similar prevalence levels when comparing the two arms. While the prevalence of hand hygiene behaviors was maintained at levels similar to the midline visit, increases in hand hygiene behaviors in the counterfactual arm narrowed the difference between the intervention and control arms at endline. Over the two follow-up surveys, there was an increase in the prevalence of household hand or facewashing stations that appeared among study arms.

• Download figure
• Open in new tab

Figure 4.

The prevalence of key sanitation, hygiene, and environmental sanitation indicators over time.]

Impacts on mental health

There was no difference between study arms in the scores for anxiety, depression, emotional distress or general well-being (Table 2). There was also no difference between the intervention and counterfactual arms in the prevalence of each mental health condition: anxiety (PR= 0.90; 95% CI: 0.72, 1.11), depression (PR: 0.83; 95% CI: 0.64, 1.07), emotional distress (PR: 0.86; 95% CI: 0.67, 1.09) and poor well-431 being (PR: 0.90; 95% CI: 0.74,1.10) (Table 2).

Secondary health outcomes

Interaction and effect modification

There was no interaction of the intervention by previous CLTSH triggering for any of the primary outcome variables of interest. We did not detect effect measure modification by sex for any of the four mental health outcomes. Similarly, we did not detect interaction by child’s sex for any of these outcomes. We also did not detect interaction between the intervention and water insecurity on any of the primary handwashing or face washing variables.

Study strengths

Our intervention was theory-informed and included an extensive intervention design process during which we emphasized the solicitation and incorporation of feedback from key stakeholders at regional, zonal, woreda, and community-levels. It was designed to be delivered at scale within the Ethiopia HEP. We utilized a randomized study design, in which intervention and counterfactual communities were allocated to treatment arms randomly. While cluster randomized trials tend to emphasize internal validity, we made considerable effort to enhance external validity. Our study was spread over three woredas in two zones, yielding a heterogeneous mix of contexts. To improve interval validity, we used a ‘fried egg’ [90] approach to select kebeles and minimize spillover. We targeted both rural and peri-urban communities and collected behavioral outcome data on a variety of household members (e.g., primary female caregiver of index child, head of household, all children aged 0-17 years).

Study limitations

The study faced significant delays in gaining local ethical approval to start the project which led to truncated implementation and follow-up periods. Key government actors were less involved than planned, which may have led to sub-optimal fidelity. The integration of Andilaye intervention activities into non-Ethiopian HEP delivery structures (e.g., hired independent community implementers) may yield further investigations into the effectiveness of the intervention on sustained behavior change and mental well-being. Several of our behavioral outcomes were reported (vs. observed), and these types of outcomes may be prone to reporting biases, indicated by differences in our reported and observed measures.