Microplanning improves stakeholders’ perceived capacity and engagement to implement lymphatic filariasis mass drug administration

Ethics statement

This project took place as an evaluation of a routine LF elimination program and was considered by Haiti’s MSPP to be a program evaluation. The project is covered by a non-research determination granted by the Center for Global Health (CGH) Human Subjects Protection Office at the U.S. Centers for Disease Control and Prevention (CDC) in Atlanta, Georgia, USA.

Project area

This evaluation took place from January through March 2018 in five of the six communes (i.e., Carrefour, Cité-Soleil, Delmas, Port-au-Prince, and Tabarre) comprising metropolitan Port-au-Prince. This area, which represents a population of approximately 2.3 million people, was the last area in Haiti to initiate LF MDA in 2012 [33].

In Port-au-Prince, MDA is delivered through distribution posts located in the community and at schools. One post, staffed by three CDDs, is allocated to treat approximately 1,000 people over a 4-day MDA. Community promoters (CPs) manage approximately three health posts and are in turn managed by community leaders (CLs). CLs, with assistance from CPs, are responsible for conducting social mobilization activities within a supervision area (SA) in the months prior to the MDA, selecting and supervising distribution sites, and training CPs and CDDs.

Overall design

To improve stakeholder engagement and MDA implementation, we conducted a 2-phase microplanning exercise in the five target communes. In phase one, LF program staff, in collaboration with CLs, conducted an inventory of the 2017 MDA distribution posts using ODK Collect (Open Data Kit Inc., v2.0) [36] loaded onto mobile devices. Historic distribution post sites were overlaid onto a Google satellite base layer (Google Inc, n.d.), using open-source QGIS (Open Source Geospatial Foundation Project, v2.14.21). To identify possible access gaps, we applied a 300 m buffer around each distribution post to identify areas within approximately a 500 m maximum walking distance. These maps were prepared to inform the discussion during microplanning workshops in phase two.

During phase two, all CLs as well as a selection of CPs, and other community stakeholders were invited to participate in microplanning workshops. In close consultation with communal and departmental focal points from MSPP, the program staff identified key community stakeholders who would be critical for community- and school-based with distribution. These individuals included school directors, Ministry of Education inspectors at the district and sub-district level. For community-based MDA this included representatives from local authorities such as mayors and CAESECs, church-leaders and pastors, voodoo temple, and local youth associations. The microplanning workshops included several key activities. Initially, the project staff provided an overview of the LF elimination program objectives, reviewed the historical results from the 2012 to 2017 MDAs, summarized the results of several key MDA evaluations, and engaged the participants in a discussion of the challenges to and opportunities for improving MDA coverage. To clarify stakeholder roles and responsibilities, program staff presented and validated the terms of references for key MDA staff (i.e., CLs, CPs, and CDDs), and drafted a document detailing expectations from opinion leaders and other stakeholders. Further, program staff presented several specific strategies that were proposed by implementing partners to improve the 2018 MDA. These strategies included but were not limited to extending the number of MDA days, modifying MDA distribution times, enhancing schools’ participation in MDA, improving visibility and credibility of CDDs, and expanding mop-up activities. Participants were invited to propose revisions including defining and validating key strategies for community drug distribution (e.g., optimal timing and location).

Finally, SA boundaries were delineated for each community leader in QGIS (Open Source Geospatial Foundation Project, v2.14.21). GPS coordinates of distribution posts collected during phase one were overlayed onto an Open Street Maps layer that included various landmarks (e.g., streets, places of business, etc.). Google Maps and satellite base layers (Google Inc., n.d.) were used as additional references for identifying landmarks and delimiting boundaries. Through an iterative process, microplanning participants used the 2017 distribution post maps as a basis to discuss and agree upon the supervision area boundaries for each CL, ensuring no overlapping or omission of areas within the targeted communes.

Impact measures

We used retrospective pre-testing to evaluate the microplanning workshops against the stated objectives [37–39]. This method involves asking participants in a single data collection event to rate their attitudes before and after the workshop. Following the microplanning workshops, participants were invited to complete a questionnaire simultaneously rating their previous year’s perceptions and their perceptions following the 2-day microplanning workshop. This strategy aimed to reduce the confounding factor of response shift bias [40], a phenomena where the respondents’ internal frame of reference is altered due to the influence of the intervention itself. In this context, the participants perceptions of their previous knowledge and engagement may be altered by participating in the workshop, as has been seen in evaluations using traditional self-reported pre-post-test evaluation frameworks[41]. Participants were asked to evaluate four measures: (i) understanding your past performance; (ii) understanding the boundaries of your coverage zones or area of influence; (iii) understanding your roles and responsibilities in the MDA; and (iv) engagement by MSPP in the MDA planning process. Participant commune and role in the MDA were also recorded. Each measure was evaluated using a 5-point scale where 1=poor; 2=fair; 3=good; 4=very good; and 5=excellent; N/A=not applicable. We also assessed changes in coverage between the 2017 and 2018 MDAs as an indirect measure of impact. MDA coverage was calculated using the total number of MDA doses delivered by each commune’s MDA distribution teams divided by the estimated population of the commune.

Statistical analysis

Data were compiled using spreadsheet software (Microsoft Excel 2010, Microsoft, Seattle, WA, USA) and data management and statistical analyses were completed using SAS v9.4 (SAS Institute, Cary, NC, USA). For pre-workshop and post-workshop summary analyses, all responses were considered, however results were filtered to include only complete pre-post pairs to measure changes as a result of the workshop. Within pair differences between the pre- and post-workshop ratings were analyzed using a Wilcoxon signed-rank test to determine whether microplanning had impacted perceived capacity to deliver quality MDA as well as key engagement metrics. Alluvial plots describing changes in participant ratings were created in R v1.1.463 (RStudio, Free Software Foundation Inc., Boston, MA, USA).

Microplanning outputs

During phase one, project staff and CLs collected 8,034 GPS coordinates representing the 2017 distribution points. In advance of the microplanning workshop, the GPS coordinates were plotted in QGIS overlayed on satellite images of Port-au-Prince to create a detailed distribution post map for all five communes. Program staff used these distribution post inventory maps to assess coverage gaps that were presented to MDA stakeholders.

During phase two, 10 2-day microplanning workshops were held for 356 participants, including 73 CLs, 240 CPs, and 43 other key MDA stakeholders selected from the five targeted communes. Several activities were conducted as part of the workshops. First, LF program staff presented workshop participants with the maps showing the location of the 2017 distribution posts. Staff then worked with CLs to delineate the 2017 SA boundaries as they were understood by each CL. Supplemental Figure 1 shows the resulting SA maps following the iterative process by which LF program staff and workshop participants modified the MDA maps to ensure no overlapping or omitted areas within the targeted communes. As part of this process, certain leaders’ distribution posts that fell outside their updated SA boundaries were flagged to be prioritized for relocation.

• Download figure
• Open in new tab

Supplemental Fig 1. Final supervision area (SA) boundaries for 2018 mass drug administration (MDA) for community leaders (CLs) by commune in Port-au-Prince, Haiti

A map generated by LF program staff in consultation with community leaders during microplanning workshops detailing supervision areas for each community leader for five communes in Port-au-Prince, Haiti. Areas are color coded by commune with shades of blue representing Carrefour, red representing Port-au-Prince, yellow representing Delmas, purple representing Tabarre, and green representing Cité-Soleil.

To support the optimal deployment of posts, each CL received a printed map following the workshop that contained detailed information about their supervision area, including roads and points of interest such as churches, businesses, and schools (Figure 1). Each CL’s supervision area was clearly outlined, and neighboring CLs were listed. Maps were printed in Haitian Creole to improve the ease of use by stakeholders and listed the CL’s name to foster a sense of ownership over the activities in that area.

• Download figure
• Open in new tab

Fig. 1. Illustrative supervision area (SA) maps, 2018 mass drug administration (MDA) in Port-au-Prince, Haiti

Illustrative supervision area (SA) map displaying the SA for a community leader in the commune of Tabarre for the 2018 mass drug administration for lymphatic filariasis (in Haitian Creole). The map includes the boundaries of the SA (in red) including infrastructure and key points of interest obtained from OpenStreetMaps. Additionally, it shows the community leaders working in adjacent SAs. For privacy reasons, this figure has been lightly edited to anonymize the names of community leaders.

Figure 1 shows an illustrative CL’s supervision map.

Another key activity included reviewing past MDA performance, identifying local challenges to achieving high coverage, and proposing updated strategies to improve program coverage. As part of this process, participants discussed and finalized distribution strategy modifications including some modifications proposed by LF program staff, as well as drafts of new social mobilization strategies and tools that were developed by LF program staff and partners. Table 1 describes a selection of key discussion points and associated solutions based on these discussions. Participants identified key actions to improve partnerships, planning and logistics associated with MDA planning; strategies to improve support for CLs, CPs, and CDDs; new social mobilization strategies; as well as modifications to improve distribution strategies.

Several challenges were identified that were unable to be addressed fully by the LF program at the time of the activity (e.g., participants from multiple communes noted a need to improve working conditions for the leaders to achieve better results, noting a particular need for umbrellas at distribution posts and increasing MDA stipends). Participants from Carrefour noted that lack of health facilities in rural sections of communes pose a challenge for adverse events management. Participants from Cité-Soleil noted the need to combat LF at different stages in the transmission cycle, especially as effluence from factories creates sanitation concerns within the commune. Finally, the LF program staff presented and validated the roles and responsibilities of CLs, CPs, and CDDs.

Respondent evaluation

Following the workshop, participants simultaneously evaluated their perceptions of their capacity and engagement prior to and following the microplanning workshops using a retrospective pre-test model. Evaluation questionnaires were completed by 339 (95.2%) workshop participants across all five targeted communes, including two (0.6%) national-level MSPP staff, two (0.6%) departmental-level MSPP staff, 68 (20.1%) community leaders, 227 (67.0%) community promoters, and 40 (11.8%) community members (Table 2). Participant counts were approximately proportional to the underlying commune population.

At baseline, understanding roles and responsibilities had the highest score 3.91 ± 0.91 (mean ± standard deviation, SD), followed by perceived engagement 3.41 ± 1.07, SAs 3.29 ± 1.12, and past performance 3.05 ± 1.00 (Table 3). The highest proportion of individuals providing a “poor” assessment was seen within respondents’ understanding of the boundaries of supervision areas (n=8, 21%), while the highest proportion of individuals reporting “excellent” was seen with understanding roles and responsibilities (n=73, 28%).

When assessing the participants’ scores after the microplanning session, the elements maintained the same rank order as in in the pre-workshop for the mean score for each metric, which were in descending order (post-test mean ± SD): roles and responsibilities (4.61 ± 0.72), perceived engagement (4.45 ± 0.79), SAs (4.61 ± 0.60), and past performance (4.37 ± 0.72).

Figure 2 shows the evolution of pre-post responses for each respondent, among complete pairs.When comparing pre-and post-workshop scores, statistically significant improvements were seen in scores across all four evaluation metrics (p<0.001). The smallest improvement was seen among respondents’ understanding of their roles and responsibilities with 0.71 ± 0.95, while the greatest improvement was seen in understanding of past performance with a mean improvement of 1.34 ± 1.05 (Table 3).

• Download figure
• Open in new tab

Fig 2. Changes in participant understanding and perceptions before and after microplanning workshops, Port-au-Prince, Haiti, 2018.

The alluvial charts track the individual-level dynamics in ratings before and after the microplanning workshops for four indicators: (a) understanding of past performance; (b) understanding of SAs/zones of influence; (c) understand respective roles/responsibilities in the MDA; and (d) perception of engagement by MSPP in the MDA planning process. The left of each chart displays the distribution of participants’ ratings before the workshop, while the right indicates the participant responses from after the workshop. Participants ranked their perception on a 5-point scale where: 1=poor; 2=fair; 3=good; 4=very good; and 5=excellent. The width of the band corresponds to the relative weight of persons with a particular before-after combination. The color of the band corresponds to the score at the before timepoint, with red/orange colors representing lower before workshop responses.

The following alluvial charts show the individual-level dynamics in rating before and after the stakeholder workshop. On the left of each chart, you see the distribution of rating before the workshop with 1 representing “Poor” and 5 representing “Excellent,” while the numbers on the right of the chart indicate the responses after the workshop. The thickness of the band represents the number of people giving a particular response combination. The color of the band indicates the response at the time point before the workshop; with orange/red colors representing lower before workshop responses. For example, this band shows the 25 individuals who reported having a “Fair” understanding of their past performance prior to the workshop, but an “Excellent” understanding of past performance following the workshop.

MDA coverage

Based on results of the microplanning meeting, the 2018 MDA campaigns in all five communes were extended from 4 to 5 days. MDAs were held from April 26 to June 4, 2018, sequentially in Tabarre, Cité-Soleil, Port-au-Prince, Carrefour, and Delmas. Reported MDA coverage increased in all five communes between 2017 and 2018 respectively with an increase from 50% to 120% in Tabarre, 50% to 93% in Cité-Soleil, 38% to 81% in Port-au-Prince, 37% to 72% in Carrefour, and 45% to 70% in Delmas.

Microplanning outcomes

These evaluation results suggest that microplanning led to several important outcomes. First, the lack of clearly defined and communicated supervision area boundaries was a central challenge to ensuring broad access to the MDA in previous years. Using microplanning, program staff were able to collaboratively define SA boundaries with all key stakeholders, thus minimizing likelihood of missing underserved areas, efficiently allocating scare MDA resources, and avoiding duplication of efforts of a limited workforce across the Port-au-Prince region. With this knowledge, stakeholders were able to collaboratively realign SA boundaries and commit to deploying posts within their respective SAs. The use of digital mapping tools was critical for helping MDA stakeholders to visualize the targeted MDA area and harmonize boundaries of their SA, without which harmonizing SA borders would have been extremely challenging in this dense urban setting. Zonal maps provided needed documentation that allowed CLs and CPs to plan and execute activities in their assigned areas, as well as more effectively coordinate with staff in neighboring areas. The presence of numerous landmarks in urban settings and orienting microplanning participants to their individual zone helped with the uptake and use of the maps by CLs and CPs.

Second, the participatory workshops provided a forum for engaging various stakeholders in identifying key MDA challenges and collectively designing locally appropriate, feasible and acceptable solutions. For example, MDA staff from Port-au-Prince commune revealed that they did not cover certain areas within the commune due to safety concerns. One such area was an economically disadvantaged and underdeveloped area. In light of this, a CL from Cité-Soleil volunteered to cover these areas to ensure that no service gaps existed. Furthermore, volunteers from the Delmas commune expressed frustrations of being scheduled as the final commune to undergo MDA in the area as well as concern that CLs from neighboring communes placed posts within their commune boundaries. They feared that these factors artificially reduced their coverage figures, as members of the population were already treated by neighboring communes. These concerns were supported by coverage data showing progressively decreasing coverage figures as the MDA progressed sequentially across communes. The refinement of SA boundaries and creation of zonal maps reduced the risk of cross-commune post deployment.

Third, the microplanning activities provided previously unknown data and information that was helpful in supporting other MDA strengthening initiatives. For instance, the distribution post census allowed for the generation of unique identification codes for each distribution post. These codes were then used to track post-based distribution data through real-time data collection, which permitted MDA staff to track the daily progress of MDA in near real time and adapt MDA procedures to achieve adequate coverage.

Finally, one of the most successful outcomes of the microplanning was an increase in community engagement, program ownership, and motivation of MDA volunteers and stakeholders. By working with CLs and other stakeholders to review distribution post distribution and refine SA boundaries, program staff sense that stakeholders felt they were involved with the “highly technical work” of MDA planning, rather than passive actors. For example, several leaders took an initiative to conduct informal, uncompensated mop-up activities beyond their allocated MDA days. Increased motivation was corroborated by the results of the pre- and post-workshop surveys from participants attending the microplanning workshop. The survey data provided additional insights into the perceived utility of such participatory planning efforts from the perspective of MDA stakeholders. For example, although LF program staff perceived themselves to be generally successful in conveying the MDA roles and responsibilities to the various MDA actors, they reported further improvements after the microplanning exercise when roles and responsibilities were reviewed and validated collectively.

The workshop evaluation supported research by Wodnik et al. with CLs, CPs, and CDDs in Tabarre and Carrefour communes [35]. Their evaluation suggested that the LF program needed to improve communication and feedback of the outcomes of the MDA rounds with the stakeholders, as well as provide opportunities for collectively identifying and implementing strategies to improve MDA [35]. Further, their evaluation revealed that SAs were poorly understood by CLs, likely leading to missed populations during previous MDA activities. Finally, they suggested that strategies such as microplanning can be a successful approach for engaging MDA staff and can contribute to increased program ownership and improved MDA outcomes.

We believe the experience in Port-au-Prince highlights several considerations that programs seeking to increase MDA coverage though microplanning in other settings, particularly complex urban locations, should consider. The strategy employed in this project was tailored to Port-au-Prince and based on known challenges in the area and considered available local resources and needs. We trust the ability to follow up the microplanning exercises with modified program strategies, such as developing new social mobilization tools or distribution methods, in response to the results of the exercise was essential. However, this requires sufficient time is available between the microplanning exercise and the MDA in order to implement these changes.

The exercise also highlighted the importance of programs’ challenging their assumptions about the reasons for low MDA coverage. The microplanning strategy uncovered barriers to MDA compliance that were unexpected and previously unknown challenges. This allowed stakeholders to leverage local knowledge to identify solutions that were acceptable to MDA volunteers and targeted communities. This is consistent with findings from other studies that have suggested that such grassroots engagement can yield a better understanding of the MDA, lead to a shared management of resources, and responsibilities, and can ultimately lead to higher treatment coverage [13, 42].

This evaluation had some limitations. Since Port-au-Prince is a large and complex urban setting, it was not possible to engage all relevant actors in the microplanning workshops (e.g., private medical sector who are influential on many individuals’ health decisions making and management of gated communities who determine the ability to access these communities). The microplanning evaluation surveys were limited by self-reporting of performance and engagement indicators, and hence, we cannot assert that microplanning led to increases in specific knowledge or performance metrics as microplanning was included as a broader MDA improvement initiative. Thus, it is not possible to attribute improvements in 2018 MDA coverage to any one improvement strategy. However, the increase in MDA coverage in all five communes in 2018 after years of gradual decline and the use of microplanning results in other MDA improvement strategies supports our belief that microplanning played a key role in improving MDA drug coverage in Port-au-Prince in 2018. Coverage was highest in the earliest treating communes and decreased sequentially, which we hypothesize was due to individuals from neighboring communes being treated and highlights the challenges of assessing coverage during post-based MDA in complex urban settings.

We believe the results of this evaluation support that microplanning should be considered in areas that are undergoing MDA for NTDs, particularly areas that have had historic challenges in achieving adequate MDA coverage. Microplanning may be particularly useful in areas where it is challenging to define the boundaries of work areas, and where the program has faced challenges in providing feedback to MDA volunteers and staff. Finally, it is important to note that microplanning is an iterative process and is ideally employed over successive rounds of MDA.