Impact of health system strengthening on delivery strategies to improve child immunization coverage and inequalities in rural Madagascar

Health system data collection

Data on monthly immunization rates from 2014 to 2018 was obtained from all 19 primary health centers in Ifanadiana district. Two health centers that were recently built and lacked consistent data across the study period were excluded. Data was obtained on all recommended vaccines in the Madagascar EPI, which included tuberculosis (BCG), measles, polio, and the combined vaccine for diphtheria, tetanus and pertussis (DTP). For polio and DTP, only the number of third doses administered was considered. Immunization information was derived from the health centers’ monthly reports to the district, which are aggregated from the health centers’ registers every month by MoPH staff. From these, the number of children immunized per month for each of these vaccines was obtained for each health center (CSB1 or CSB2), which included all children vaccinated through both routine services and outreach activities. The population of children aged 12-23 months was also obtained for each health center catchment from official MoPH records [47]. Data quality was monitored by joint PIVOT-MoH supervisions every three months. Information on the geographic extent and timing of the HSS intervention was obtained from the NGO’s internal records.

Cohort data collection

We obtained population-level information from the Ifanadiana Health Outcomes and Prosperity longitudinal Evaluation (IHOPE), a district-representative longitudinal cohort study initiated in Ifanadiana district in 2014 [48]. It consists of a series of surveys conducted in a sample of 1600 households every two years, with questionnaires modeled after the internationally-validated Demographic and Health Surveys (DHS) [49]. A two-stage sample stratified the district by the initial HSS and control catchments. Eighty clusters, half from each stratum, were selected at random from enumeration areas mapped during the 2009 census, and households were then mapped within each cluster. Twenty households were selected at random from each cluster. 1522 households were successfully interviewed in 2014 (95.1% acceptance rate), 1514 and 1512 households were revisited during the follow-up in 2016 (94.6% acceptance rate) and in 2018 (94.5 % acceptance rate) respectively. Data collection, survey coordination and training were conducted by the Madagascar National Institute of Statistics (INSTAT).

The study was reviewed and approved by the Madagascar National Ethics Committee and the Harvard Medical School Institutional Review Board. The survey included a household questionnaire and individual questionnaires for all men and women of reproductive age (15-59 years and 15-49 years, respectively). All eligible women and men who were in the households sampled (usual residents or visitors) were interviewed. Data collected through the questionnaires included general information about household composition (size, genders, ages); living conditions, education, and other indicators of socio-economic status; recent illness, care seeking for illness and preventive behaviors; women’s reproductive history and care seeking behavior for reproductive health; children’s health, development, preventive behaviors, and care seeking for illness; and child, adult and maternal mortality. For vaccination specifically, information about vaccination status of the children under 5 years was obtained from the individual interviews with their mothers. Vaccination status and history was assessed from the children’s vaccination cards when available, or from the mother’s report otherwise.

Analysis of immunization rates at health centers

First, for each vaccine (BCG, Polio 3^rd dose, DTP 3^rd dose, and Measles), we estimated monthly per capita rates (age-specific) at each health center. We then modelled temporal changes in immunization rates over the study period, studying the contribution of health system factors, time-varying factors and the effect of programs and policies. For health system factors, we studied baseline differences between health centers in the initial HSS catchment and in the rest of the district, as well as between different types of health center (CSB1 and CSB2). For time-varying factors, we studied annual linear and seasonal changes in immunization rates in the district. Seasonal changes were studied using a sine function with a period of one year and the horizontal shift that best fitted the data. For the effect of programs and policies, we studied the effect of VWs and of the HSS intervention on immunization rates. For this, we built dummy variables coded as 1 for the CSBs and months where each program was in place (discrete for months with VWs, and constant from the moment the HSS started until the end of the study period). We also studied the interaction of the HSS intervention with linear annual change and VWs, in order to account for the additional effect of the HSS intervention over time, and for changes in the contribution of VWs to overall immunization rates due to the HSS intervention. We excluded from the analysis the measles immunizations delivered via SIAs in October 2016 because the target age was children up to 5 years of age, which differed from the population group used in the analyses (12-23 months).

Per capita immunization rates were modelled separately for each vaccine via binomial regressions in generalized linear mixed models, including a random intercept for each health centers. All other variables were included as fixed effects. Univariate analyses were first performed for each explanatory variable and those with p-value below 0.1 were retained for multivariate analysis. From this full model, a reduced model that included only variables reaching statistical significance (p-value < 0.05) was obtained via backwards selection. Effects are reported as adjusted Odds Ratios (OR).

Analysis of vaccination coverage in the longitudinal cohort

While an analysis of health center immunizations can provide some basic understanding about the impact of the HSS intervention on RI and outreach activities over time, it does not allow for obtaining accurate measures of vaccination coverage due to known inaccuracies in target population estimates. In addition, aggregated information reported by the health system does not allow us to evaluate changes in economic or geographic inequalities in vaccination coverage, or for the assessment of the timeliness of vaccination, all of which can be affected by the relative contribution of RI and outreach activities in the area. For this, we conducted a complementary analysis of vaccination coverage at the population-level using data from the IHOPE cohort.

Vaccination coverage was estimated for 2014, 2016 and 2018 from individual level data for children 12-23 months or 12-59 months (depending on the analysis, see below), as the proportion of the target group immunized at the time of the interview. Similar to our analysis of health center immunization rates, we studied separately each of the recommended vaccines, namely BCG, Polio 3^rd dose, DTP 3^rd dose and measles. We also estimated whether the child had received all of these recommended vaccines. For each child surveyed, vaccination status for each vaccine was coded 1 if the child was vaccinated based either on the vaccination cards, or on the mother’s report, and 0 otherwise. To assess the impact of economic and geographic inequalities in vaccination coverage, we estimated a household wealth score via principal components analysis of household assets following standard DHS methods [49], and we estimated the shortest path distance from the villages in each cohort cluster to the nearest health center using the Open Source Routing Machine engine. For this, we had previously mapped the entire district of Ifanadiana on OpenStreetMap, resulting in over 23,000 km of footpaths and 5,000 residential areas mapped [50]. Households were ordered based on their wealth score and distance to the nearest health center and were classified into five quantiles with 20% of observations in each category (Q5 = wealthiest or closest to the health center; Q1 = poorest or most remote). Vaccination coverage in children 12-23 months was estimated inside and outside of the HSS catchment at the beginning and at the end of the study period (2014-2018), disaggregated by wealth quantile and by distance quantile. Changes in inequalities were measured as the gap in vaccination coverage between the worst-off quantiles (Q1-Q2) and the best-off quantiles (Q3-Q5) over time.

We then modelled changes in vaccination coverage over the study period, studying baseline differences and annual changes in overall coverage and in economic and geographic inequalities for the HSS catchment and the rest of the district. For this, we performed a separate logistic regression mixed model for each vaccine, with the household cluster as random intercept. To study baseline differences between HSS catchments we included a dummy variable reflecting whether clusters where located in the initial HSS catchment. We included the natural logarithm of the wealth score to study differences in socio-economic groups, and distance to health center (in tens of kilometers) to study differences in geographic groups, both as continuous variables. We included two time-varying variables, one to reflect annual changes in vaccination coverage in the whole district, and another to reflect changes per year of HSS intervention in the HSS catchment. Finally, we included interaction terms of these two variables with wealth and distance to study the evolution of inequalities in each area. We included children aged 12-59 months in these analyses to allow for adequate sample sizes for each model. Model selection procedures were identical to those described above for the analysis of health system data. To understand which population groups could reach recommended vaccination coverage targets in the HSS catchment and in the rest of the district, we predicted in-sample vaccination coverage for 2018 from each of the reduced multivariate models, at varying levels of socio-economic class and proximity to health centers.

Finally, we studied the difference in timeliness of vaccination between the HSS catchment and the rest of the district in the subset of children 12-59 months with vaccination cards at the time of the interview in any of the cohort waves (N=786). For this, we estimated the child’s age at vaccination from the date of birth and the date of vaccination. Timely vaccination was estimated for each vaccine based on the recommended age of vaccination by the national EPI in Madagascar: in the first month of life for BCG (recommended to be given at birth), in the fourth month for Polio 3^rd dose and DTP 3^rd dose (recommended to be given the 14^th week), and in the ninth month for measles [37].

Trends in vaccination coverage and inequalities

In total, data from 2,699 children between 12 and 59 months of age was obtained from the longitudinal cohort. Of these, 651 were between 12 and 23 months old, the age at which all four immunizations studied here should be completed. Vaccination coverage for children 12-23 months was very low at baseline, ranging from about 54 to 59% depending on the vaccine. Only 34.6% of children 12-23 months were fully vaccinated in 2014. Consistent with analyses of health system data, coverage for most vaccines improved substantially during the study period, especially in the HSS catchment (Figure 2). In 2018, 63.6% of children were fully vaccinated in the HSS area, compared with only 37.5% in the rest of the district. Coverage in 2018 varied for each vaccine considered; BCG had the highest coverage (80.8% inside and 70.3% outside the HSS catchment), whereas measles had the lowest coverage (73.2% inside and 49.2% outside the HSS catchment). The minimum recommended coverage of 90% was not reached for any of the vaccines, either inside or outside the HSS catchment.

• Download figure
• Open in new tab

Figure 2. Changes in vaccination coverage for children 12-23 months and associated inequalities in Ifanadiana District, 2014-2018.

Left panel shows changes in immunization coverage over time, split by HSS catchment and type of vaccine. Middle and right panels show inequalities in coverage over time, according to geographic quantiles (distance to health center) and economic quantiles (wealth score), respectively. Color gradient shows the difference in coverage between the best-off groups (quantiles 3 to 5) and the worst-off groups (quantiles 1 and 2), from dark blue (greater difference, more inequalities) to light blue (smaller difference, less inequalities).

Disparities in immunization coverage were observed according to households’ geographic distance to health centers and wealth, with different trends in the HSS catchment and in the rest of the district (Figure 2). In 2014, the difference in coverage between households living closer (quantiles Q3-Q5) and further (Q1-Q2) from health centers ranged from 25% to 32%, except for measles vaccine. Differences between wealthier (Q3-Q5) and poorer (Q1-Q2) households were smaller, between 5 and 15% for most vaccines. After four years, economic inequalities in vaccination coverage were substantially reduced in the HSS catchment, with little change in geographic inequalities. In contrast, in the rest of the district geographic inequalities were greatly reduced, while economic inequalities increased for all vaccines except for polio. Table S2 shows vaccination coverage rates in each of the cohort years (2014, 2016, and 2018) and different population groups.

Determinants of vaccination coverage trends and predictions of coverage targets

Multivariate analyses of vaccination coverage trends between 2014 and 2018 revealed consistent predictors for most of the vaccines studied (Table 2). Baseline differences between the HSS catchment and the rest of the district were observed for only two vaccines, BCG (OR=0.6) and DTP (OR=1.65). Coverage of each of the four vaccines was positively associated with household wealth and negatively associated with household distance to health centers. The odds of vaccination for children in remote households was between half (OR=0.52, measles) and a fifth (OR=0.22, BCG) for every additional 10 km from the nearest health center. Vaccines with three required doses were the most associated with household wealth, with an odds ratio of 2.58 for DTP and 2.85 for polio. District-wide improvements in vaccination coverage were associated with a reduction in geographic inequalities over time and not with a homogeneous improvement for all population groups. Indeed, the OR of the interaction between annual change and distance to health center ranged from 1.17 (all vaccines) and 1.31 (BCG and polio), meaning that each year households living far from health centers had progressively better coverage, closing the gap with those living in close proximity. Changes in the HSS catchment were distinct from the rest of the district. First, every year of HSS intervention was associated with an increase in the odds of vaccination in the HSS catchment between 1.18 (measles) and 1.43 (BCG), except for DTP. Unlike the rest of the district, children from wealthier households in the HSS catchment had lower odds of vaccination over time (OR of interaction ranging from 0.73 to 0.83). However, the decrease in the odds of vaccination over time for more remote households in the HSS catchment (OR of interaction ranging from 0.72 to 0.84) effectively compensated the positive effect observed in the district as a whole, meaning that geographic inequalities were only reduced outside the HSS catchment.

In-sample predictions from these multivariate models for 2018 revealed stark differences for achieving international coverage targets depending on HSS support and population characteristics (Figure 3). Overall, a 90% coverage (recommended coverage at the national level) could only be achieved for BCG, and just for populations who live in close proximity to a health center with HSS support and who are among the wealthiest in the area. When the target is relaxed to 80% coverage (minimum coverage recommended for every district), there were some population subgroups for which this target could be achieved in the HSS catchment for every individual vaccine. The range of socio-economic and geographic groups for which minimum coverage rates could be reached was much larger for BCG and Polio than for DTP and measles (Figure 3). Coverage targets for all recommended vaccines simultaneously (instead of each independently) could not be achieved for any population group. In areas outside of the HSS catchment, a 90% coverage was not achieved for any of the recommended vaccines or population subgroups. Only those in the top percentiles of wealth and proximity to a health center achieved an 80% coverage for BCG vaccination without HSS support.

• Download figure
• Open in new tab

Figure 3. Predictions for achieving vaccination coverage targets for different population groups in Ifanadiana District.

Graphs show in-sample predictions of vaccination coverage for the year 2018, estimated from models fitted with the cohort dataset (coefficients available in Table 2). For this, vaccination coverage was estimated for every combination of household distance to health center and wealth (split into deciles) in the HSS catchment and in the rest of the district, using the fixed effects of each model. Areas with predicted coverage greater than 90% or 80% are surrounded with white dashed lines or black dashed lines, respectively.

Timeliness of vaccination

Among the 786 children aged 12-59 months who had a vaccination card at the time of the interview (of children), timeliness of vaccination varied widely depending on HSS support and the vaccine considered (Figure 4). Most children were vaccinated in the first month of life for BCG, at 4-5 months for the third dose of polio and DTP, and at 9-10 months for measles (Figure 4A). Vaccination occurred later than recommended in national policies (see methods section) for many children, especially outside the HSS catchment. As a result, the proportion of children vaccinated at the recommended age was higher in the HSS catchment, ranging between 58% for BCG and 44% for polio and DTP (Figure 4B). In the rest of the district, this proportion was significantly lower and ranged between 49% for BCG and 22% for polio and DTP. Timeliness of vaccination improved between 2014 and 2018 in the HSS catchment for all vaccines except for BCG, while it only improved for measles in the rest of the district (Figure S1).

• Download figure
• Open in new tab

Figure 4. Timeliness of vaccination among children 12-59 months in Ifanadiana District.

A) Age of vaccine administration (in months) as reported in children’s vaccination cards inside the HSS catchment (solid lines) and in the rest of the district (dashed lines), with colors representing each type of vaccine. B) Proportion of children vaccinated at the recommended age for BCG (1^st month), third dose of Polio (4^th month), third dose of DTP (4^th month) and Measles (9^th month), as per the Madagascar Expanded Program on Immunization.