1. Abstract
To address the current threat of antimalarial resistance, countries need innovative solutions for timely and informed decision-making. Integrating molecular resistance marker surveillance into routine malaria surveillance in pre-elimination contexts offers a potential early warning mechanism to identify resistance signals and target areas for further investigation, including therapeutic efficacy studies. However, there is limited evidence on what influences the performance of such a system in resource-limited settings.
From March 2018 to February 2020, a sequential mixed-methods study was conducted in primary healthcare facilities in a South African pre-elimination setting to explore factors influencing the flow, quality and linkage of malaria case notification and antimalarial molecular resistance marker data. Using a process-oriented framework, we undertook monthly and quarterly data linkage and consistency analysis at different levels of the health system, as well as a survey, focus group discussions and interviews to identify potential barriers to, and enhancers of, the roll-out and uptake of this integrated information system.
Over two years, 4,787 confirmed malaria cases were notified from 42 primary healthcare facilities in the Nkomazi sub-district, Mpumalanga, South Africa. Of the notified cases, 78.5% (n=3,758) were investigated, and 55.1% (n=2,636) were successfully linked to their Plasmodium falciparum molecular resistance marker profiles. Five tangible processes—malaria case detection and notification, sample collection, case investigation, analysis and reporting—were identified within the process-oriented logic model. Workload, training, ease of use, supervision, leadership, and resources were recognized as cross-cutting influencers affecting the program’s performance.
Approaching malaria elimination, linking molecular markers of antimalarial resistance to routine malaria surveillance is feasible. However, cross-cutting barriers inherent in the healthcare system can determine its success in a resource-limited setting.
2. Introduction
Sub-Saharan Africa bears the largest burden of malaria, with 94% and 95% of the global cases and deaths, respectively.[1] Malaria-endemic countries in Sub-Saharan Africa are now also facing the looming threat of antimalarial drug resistance. Malaria parasites with reduced susceptibility to artemisinin-derivatives are emerging and rapidly spreading, threatening the continent’s control and elimination goals, heightening the need for novel tools and strategies to effectively tackle this threat.[2–8] Such innovations need to be timely, relevant and tailored to existing health systems, particularly in resource-limited settings.
In its strategy to respond to antimalarial drug resistance in Africa, the World Health Organization (WHO) emphasised the need for robust and agile surveillance systems capable of promptly detecting and responding to antimalarial drug resistance and recommends that this surveillance be integrated into routine malaria surveillance systems.[9,10] This integration is particularly important in pre-elimination areas, where the risk of drug-resistant parasites emerging may be heightened by higher drug pressure and lack of partial immunity.[11] In these settings, the WHO also recommends monitoring the prevalence of molecular markers of antimalarial drug resistance, as an early warning system to identify resistance signals and areas requiring further investigation, including therapeutic efficacy studies (TES). Despite TES being the cornerstone of antimalarial drug efficacy monitoring, they are not conducted regularly and do not provide a timely spatial representation of the distribution of antimalarial resistance. Previous evidence has shown that integrating antimalarial molecular markers using a routine malaria notification system is feasible in sub-Saharan African settings.[12] However, little is known about the barriers and enhancers to integrating molecular markers of resistance within the routine malaria surveillance system. Understanding such factors is needed by countries adopting or expanding molecular surveillance, for which recent laboratory investments have been extensive.[13,14]
In 2018, the South African National Malaria Programme (NMP) piloted a novel technology to consolidate and enhance malaria surveillance activities and treatment approaches fundamental to achieving malaria elimination. Through the Smart Surveillance for Malaria Elimination (SS4ME) initiative, antimalarial drug resistance molecular marker data were linked to malaria case notifications in near real-time in Nkomazi, Mpumalanga, a pre-elimination setting.[15] SS4ME included the collection of malaria rapid diagnostic tests (mRDTs) and, wherever possible, dried blood spots (DBS) on filter papers for assaying molecular markers of antimalarial resistance that can be linked by unique barcodes to individual case notifications.
Developing and evaluating the integration of molecular markers of antimalarial resistance into routine malaria surveillance required an assessment of the surveillance system’s performance, both independently and together with other routine notification components, making for a complex intervention.[16] As per national malaria treatment guidelines, all suspected malaria cases presenting to health facilities should be confirmed by a mRDT or microscopy (passive case detection) before treatment is administered.[17] Additionally, in pre-elimination areas of South Africa, the NMP screens high-risk groups, such as migrant and mobile populations (proactive case detection), and households surrounding the residence of index cases (reactive case detection).[6,7] Therefore, integration of molecular resistance markers into routine malaria surveillance system involved adopting several technologies, activities, and processes into the existing malaria notification system, with regular monitoring and adaptations. For SS4ME, a conceptual framework was developed to explore and guide how the roll-out, adoption and utilisation of new and existing technologies would inform malaria elimination goals. This laid a foundation for benchmarking how SS4ME was received and understood by potential key stakeholders. As with other such conceptual theories, this framework lacked a mechanism to explore the interaction between the different determinants of success and how these interact with the users or beneficiaries. Therefore, this approach needed to be revised and expanded to identify internal, external and interactive factors that could affect the implementation and impact of the intervention. Through the step-by-step depiction of how a program operates and how it leads to the desired outcomes, process-oriented logic models have been used in programme planning and evaluation to understand its key elements.[18]
‘Making Data Mapworthy’ was a quantitative sub-study linked to the SS4ME pilot that evaluated the feasibility of integrating molecular resistance markers into the routine malaria surveillance system using coverage, accuracy and linkage of malaria cases in near real-time.[12] However, these metrics alone could not fully examine the flow of data, or users’ practices and perceptions. Here, factors influencing the flow, quality and linkage of malaria notification data and associated molecular resistance markers from different reporting levels are investigated to inform the enhancement and sustainability of this integrated early warning system for antimalarial resistance.
3. Materials and Methods
This SS4ME sub-study used an iterative sequential mixed-methods design and included:) monthly and quarterly quantitative descriptive analyses, 2) a healthcare facility staff survey, and 3) focus group discussions (FGDs) and in-depth interviews (IDIs) with healthcare facility staff involved in malaria case management and NMP staff in Nkomazi Sub-District, a pre-elimination area in Mpumalanga province, South Africa. Individuals identified through proactive, reactive, or passive case detection were screened for malaria using a falciparum-specific histidine-rich protein 2 (HRP2)-based mRDT. Both symptomatic and asymptomatic positive cases were treated following national guidelines with the WHO-recommended 3-day artemether-lumefantrine regimen, used in the area since 2007. For molecular surveillance, DBS filter papers were to be collected from all patients with positive mRDTs. Demographic and case data were collected through the Notifiable Medical Condition form or app, and verified for quality within 24 hours at the sub-district NMP office. Case investigators conducted household visits within 24-72 hours of notification, recording Global Positioning System (GPS) coordinates and assessing malaria risk factors. All notifications were quality-checked and electronically captured into the routine District Health Information System II (DHIS2) at the sub-district NMP office.[17]
Three levels of notification data were examined: healthcare facilities, the sub-district NMP office serving as a data collection / capturing centre and provincially through DHIS2 submissions. All SS4ME-participating healthcare facilities were enrolled for malaria case data analysis at monthly and quarterly intervals, to gauge coverage and consistency as performance metrics. The facilities were then categorised as low- or high-performing on the overall data linkage, and two from each level were purposively selected for enhanced data quality assessment.
A paper-based survey (Tool S1) was administered between 01 March and 30 June 2020 to primary healthcare facility staff treating malaria patients to evaluate their practice, perception, and experience of the integration of molecular resistance markers into routine malaria surveillance system activities. For FGDs and IDIs, various cadres of staff involved in malaria case management at healthcare facilities and in the NMP were invited to participate by email or phone call. At least one staff member performing any malaria-related activities was invited per healthcare facility. After obtaining verbal and written consent using Tools S2 (b) and (c), FGDs and IDIs were conducted in English from 3 – 6 June 2020 using pre-prepared interview guides (Tools S3 and S4) to maintain consistency and quality. The audio recordings were securely stored on the password-controlled study computer and later transcribed. Two study investigators listened to audio recordings with reference to the transcribed scripts and resolved any interpretation conflicts or transcription errors by consensus.
This study was approved by the Human Research Ethics Committee of the Faculty of Health Sciences at the University of Cape Town (HREC REF Number: 698/2019 and 038/2020 (Document, S1 and S2), including for data obtained in the SS4ME study (HREC REF Number: 519/2017 [Document S3]). SS4ME was also endorsed by the South African Department of Health, the National Institute for Communicable Diseases (NICD) and the Mpumalanga Malaria Programme. All partners were notified, and their staff were informed of the study, including NMP staff, information officers, clinicians and data clerks.
The descriptive analysis focused on staff survey data and monthly and quarterly DHIS2 data evaluations. The latter summarised notified and investigated cases, mRDT / DBS samples analysed, their linkage, as well as spatiotemporal trends in molecular markers and usability assessments. All quantitative data analyses were conducted using R programming language (versions 3.6 and 4.0). Further quantitative methods used have been explained elsewhere.[12] For consistency, quarterly aggregates of notified cases and investigated cases from each healthcare facility sampled were compared at three levels (healthcare facility, sub-district data capture centre and provincial DHIS2 records) and the median difference was computed. Consistency was defined as an equal number of cases being reported at different levels, with a difference of +/− 5 cases allowed to account for delayed reporting. Since a consistency benchmark for integrating molecular resistance data into malaria case data had not been described before, this was established by adapting the internal consistency benchmark proposed by the WHO Data Quality Assurance guideline.[19] Data were then explored quantitatively and qualitatively to identify possible causes of inconsistency.
For the qualitative data, each audio recording was transcribed and imported to NVivo 12 before being coded deductively, based on the conceptual framework and interview guides, and inductively from other observations. The first coding cycle assigned labels to text excerpts in the transcripts as lowest level ‘nodes’. These were then explored for repeated ideas and patterns, which were grouped, organised and categorised as higher-level themes, based on the interview guide sections and new concepts emerging. The prevailing themes and their theories were extracted and further analysed through the framework matrix, and the analytical outputs were evaluated across the range of participants and stakeholder groups.
As a complex intervention, the study used a process-oriented logic model to link the overall qualitative and quantitative data to determine the enablers and barriers that would explain the changing trends. To do so, results were further categorised into processes and logically analysed to identify inputs and activities, intended and unintended outputs, and outcomes. A matrix was constructed to explore the level of influence and performance for each key feature identified from the process logic model.
4. Results
A total of 4,787 malaria cases were notified in Nkomazi, Mpumalanga with 78.5% (n=3,758) investigated by the NMP (Fig. 1) from March 2018 to February 2020. Of the notified cases, 55.1% (n=2,636) were linked to their Plasmodium falciparum molecular marker resistance profiles, with 85% (n=2,240) of the linked cases mapped to healthcare facility, ward and locality levels. Further quantitative results are reported elsewhere.[12]
Summary of cases notified, investigated, and linked to molecular resistance markers and locality from March 2018 to February 2020 in Nkomazi sub-district, Mpumalanga.
A total of 46 healthcare workers participated in five separate FGDs, with group sizes ranging from 8 – 12 participants. Of the participants, 32 were nurses (including professional nurses, occupational nurses, and assistant nurses), and 14 were environmental health officers who worked as health promoters or malaria case investigators, data clerks, data capturers and surveillance officers. The majority of the participants were women (n=31). Four IDIs were conducted with staff at the supervisory and decision-making levels performing infection control coordination, district and provincial NMP management, primary healthcare management and malaria surveillance supervision.
The 50 participants involved in the FGDs and IDIs had between 1 and 31 years of experience working with the NMP and were involved in the surveillance data reporting chain. Their responsibilities included field and clinical diagnostics, care of patients, case notification, case investigation, data capture, sample collection, labelling and transport, supervisory and reporting roles and decision-making. Several themes emerged from discussions on the integration of molecular resistance markers into the routine malaria surveillance system, which included malaria case notification, sample collection, packaging and transportation, case investigation, data capture and reporting.
A total of 64 nurses from 21 of the 42 healthcare facilities participated in the survey. Respondents were registered / professional nurses (67.1%), enrolled nurses (22%) and occupational nurses (6%). Two respondents did not mention their job titles. Five quarterly assessments were conducted for three healthcare facilities for the consistency evaluation. Due to logistical challenges, the team did not manage to enroll the fourth health facility.
In Fig. 2 below, prevalent themes in the FGDs and IDIs included ease of use, perceived usefulness of surveillance, staff members’ reluctance to adopt new activities, contradictions in best practice definitions, workload, and system support. These themes converged into sub-themes: work commitment, agency and ownership, challenging processes, compromise, staffing needs, training, leadership, and supervision.
The illustration of factors discovered during focus group discussions and in-depth interviews influencing the implementation of molecular resistance markers into the routine malaria surveillance system in the Nkomazi sub-district.
Several themes and sub-themes surfaced from these discussions. Initially, participants voiced reluctance and contradictions, leading to confusion over roles. This uncertainty spawned a challenging start and negatively impinged on the work. To counter these issues, more intensive training, stauncher leadership, and consistent supervision were recommended. In instances where confusion persisted, a supportive system was seen as beneficial to harmonise tasks and provide necessary training and quality control. This tactic successfully alleviated immediate challenges. Lastly, even though the integration process demanded an increased workload, healthcare staff who were willing to learn and perceived the benefits of this integration displayed a heightened commitment to their work. This level of dedication indicates a need for increased staffing or a reorganised work system to provide adequate support.
Five key processes (Table S1) emerged: malaria case detection and notification, sample collection, case investigation, data capture, analysis and reporting. These themes are described below using the process-oriented logic model outlined in Table S1, linking them to processes, inputs, outputs, and outcomes.
4.1 Process I: Malaria case detection and notification
The change introduced during the integration of molecular resistance markers into the routine malaria surveillance system was the inclusion of barcode stickers on the case notification forms during passive, proactive and reactive case detection. The study also developed activities to enhance the collection of all required malaria case details in the reporting notification form (e.g. contact information and household address / directions). This was to support the linkage of demographic and location data to the molecular markers of resistance. This also resulted in additional workload, which was aggravated by the multiple notification reporting systems. As shown in Fig. S1, the NMP introduced two new malaria notification systems, leading to three malaria notification systems running concurrently during the study period, namely a) the Notifiable Medical Condition (NMC) book / forms, b) the Malaria Connect mobile application and c) NMC mobile application.
The surveillance team supervisors mentioned that healthcare workers routinely used the Malaria Notification Book for notification, and only notified cases using other systems as an additional option. This was corroborated by healthcare workers themselves who reported using the notification book first and other notification methods depending on the workload, as quoted below:
‘We do the paper first, the notification book first, the remaining depends on their time or workload’. [Nurse, IDI06].
However, in responding to the survey question on which systems are used frequently for notification, 61/64 participants reported using these interchangeably: Malaria Connect (36, 59%), NMC Notification Book (15, 25%) and NMC Mobile Phone Application (3, 5%). Of those who reported using more than one system, only one (2%) reported using all three, while five (8%) used Malaria Connect and NMC Notification Book only, and one (2%) used both Malaria Connect and NMC Phone App (Figure 3).
Malaria notification systems used by 61 staff working in 21/42 healthcare facilities in Nkomazi, Mpumalanga, South Africa. NMC: Notifiable Medical Condition.
Participants reported that the parallel reporting systems introduced an additional burden to their daily work, particularly in the community health centres (CHC) operating 24-hours per day, 7 days a week, where a high number of patients present. For example, one of the nurses from a CHC in the FGDs said:
‘But the volume you can look at the CHCs [Community Health Centres] and eight-hour [clinics], there are more people coming in because day and night we are working.’ [Nurse FGD03].
4.2 Process II: Sample collection
The project introduced the collection of DBS filter papers, with related SOPs at all participating healthcare facilities. Participants expressed reluctance about collecting DBS and capturing patient information on mRDTs. This called for increased programmatic support and training, and with time, the study observed improvement of barcoded mRDT samples received at the NICD laboratory from 19% in the first quarter to 85% in the final quarter, reflecting gradual acceptance and adaptability. [12]
‘Basically, at first it was quite a daunting task because now [I] remember we had facilities that recently introduced the new NMC form and also Malaria Connect and all of that. So when we went to train the health facilities [about notification and DBS sample collection] they were resistant because they feel that it is extra work on their behalf…’ [Environmental Health Officer, FGD01].
Not all healthcare workers grasped the procedure for blood collection immediately. Due to difficulty in obtaining enough blood from the initial finger prick, they occasionally had to prick the patient’s finger a second time. This caused some resistance among patients, according to the staff:
‘Yes, there is not enough [blood from one finger prick]. There is a little, but it is not enough. So, most of the time they [nurses] have to prick maybe twice, or three times sometimes.’ [Case investigator, FGD01].
However, this also encouraged the healthcare workers to understand the rationale behind filter paper sample collection, to be able to explain this to the patients who were hesitant:
‘These activities have helped us to know what we are doing and to help the patients; if they ask me now why we are taking another blood sample, it is for monitoring drug resistance and making sure you are cured. So, it’s a very good thing’ [Nurse, FGD04].
After DBS sample collection, healthcare workers were asked to label and package samples, ready to be collected and transported each week by the NMP to the national laboratory. The intended output was to obtain accurately barcoded filter papers for molecular analysis and linkage to individual case notifications. However, this again meant an additional workload for the healthcare staff, which might have a negative impact on their other activities. In the first quarter, poor sample collection, packaging and recording of the required patient details, large numbers of samples / notification forms without barcodes and a low number of quality samples shipped to the central lab for molecular analysis were observed. This necessitated frequent refresher training, quality checks and supportive supervision. Despite the slow start, the overall outcomes of the number and quality of samples for molecular marking increased, for instance, the overall linkage of the molecular resistance markers and case notification rose from 51% in the first quarter to the highest of 75.1% in the third quarter, with participants noting how the activities strengthened the routine antimalarial resistance surveillance.[12]
‘… remember we went with the supervisor to a couple of clinics when we identified that problem (positive mRDT sample with insufficient or missing DBS) and then we went back and told them that now we no longer just take the RDT. We also need to take the dry blood spots, so we need enough blood for the dry blood because sometimes we find that the dry blood is just a tiny bit of the blood and it really cannot be used. So we had to make sure that we go back to them and tell and train them that when you prick, make sure you prick enough just in case it is a positive you can also get more blood. They are currently a lot better’ [Nurse, FGD03].
As shown in Table 1, among the 64 survey participants who performed these tasks, the majority reported that overall, it is easy for them to collect, label, package and ship DBS samples (55.1%). However, 29.5% found it difficult sometimes, and 9.4% found it always difficult.
4.3 Process III: Case investigation
Malaria case investigators aim to trace all confirmed malaria cases, obtain their residential GPS coordinates, identify malaria risk factors and actively screen nearby contacts for malaria (reactive case detection). The additional intended output for the integration of molecular resistance markers into the routine malaria surveillance system was to enhance residential GPS coordinate data quality using eTrex - 10 GPS devices or Samsung tablets, with training and related SOPs.[20] Overall, 78.5% of malaria cases were successfully investigated (Fig. 1), with the accuracy of GPS coordinates increasing from 48% in the first quarter to 89% in the last quarter.[12] The poor accuracy in the first two quarters was associated with device-related (non-uniform settings, device malfunctioning, lack of enough battery) and human-related (transcription errors, incomplete patient addresses) and system and resource-related factors (lack of enough battery or backup devices, increased workload, formatting during file transfers).
While discussing the issue of missing contact details, participants in FGDs mentioned that many of their patients are immigrants who lack phone numbers, travel far to seek healthcare or do not know how to clearly describe the locations of their recently acquired residences. Additionally, due to their foreign and, at times, undocumented migrant status, many patients are often hesitant to share their contact details. These missing details lead to case investigation failure, resulting in more missing data.
The case investigators occasionally reach out to the notifying healthcare staff when faced with a case lacking contact information, aiming to gather additional details about the patient. This additional follow-up was received both positively and negatively. Positively, this would reduce the cost of case investigation since they travel substantial distances for case investigation, and a few more details can assist in finding the index case’s location more directly.‘We drive almost 40 km or more to find a case, then the phone number is wrong and there is only one line for the address. Just looking for one case, you take the whole day while other notifications are waiting.’ [Case investigator, FGD01].
However, the practice of case investigators contacting clinic staff on their mobile phones was met negatively by some clinic staff, who perceived it as an invasion to their privacy, potentially threatening their working relationship. During a FGD a nurse stated:
‘Yes, they do so [call] because in the notification book we also write our phone numbers when we notify a patient; if there is something missing they can call. Even if we are not on duty, yah, threatening our working relationship.’ [Nurse, FGD03]
On the other hand, some clinic staff and case investigators reported that calling each other is not a problem, even after-hours calls, mentioning that it’s useful and gives them a sense of duty. They have developed trust, and without phone calls, investigators would be travelling long distances to look for cases. One participant in an FGD with the nurses said:
‘…it is fine because they want information, and they want to know if we have referred the patient [to hospital] or if the patient is going home.’ [Nurse, FGD03]
Another challenge raised by the NMP staff was administrative inefficiencies resulting in multiple notification books at health facilities. These contribute to some tension. For instance, case investigators reported having collected the notification papers from the wrong notification book (outdated book / not currently in use) at some larger facilities, where several notification books may be used simultaneously. Others failed to access the notification book during the facility visit due to it being misplaced. This is usually resolved by removing the outdated books and leaving the facility with only one notification book and informing the supervisors.
To help address these gaps, the study identified the need for regular refresher training on malaria surveillance with guidance on malaria notification and capturing of case information at the facility level, supported with new SoPs and facility guidebooks. The additional hours spent on refresher training and learning the new SoPs were rewarded when the interviews revealed that these activities resulted not only in an increase in the accuracy of coordinates as stated above, but also in an overall improvement of the NMP’s surveillance system functioning. An information officer in a FGD said:
‘The ownership thing you know when you understand exactly what it is you are capturing and how important it is, then it makes all the difference in terms of making sure it is on file, because how these things usually get presented … it is always just another study that we do not know how it is going to end up, who it is going to benefit. So, I think also that approach of you [the study team] is excellent as we know what this is … and what this is for. This is how important it is, it is ours only. [Information officer, FG02]
4.4 Process IV: Data capture
Routinely, malaria-related data are captured at the sub-district data capture centre daily. Malaria case investigators submit case report forms from both the healthcare facilities and their case investigation visits. The additional data input for the integration of molecular resistance markers into the routine malaria surveillance system was the capturing of the barcode number into the DHIS2 system. A barcode scanner was introduced to reduce workload and avoid transcription errors. Even after being trained, data clerks experienced challenges using the scanners, leading to inconsistencies in barcode capturing with both manual capturing and scanner use. The barcode scanning was also affected by internet connectivity, as any drop in WiFi signal disconnected the DHIS2 system. Overall, the process increased the workload and affected the perceived usefulness of the scanner, quoting a data clerk interviewed:
‘Typing is easier than using a barcode scanner’. [Data clerk, FGD01].
Furthemore, delays in data capturing could last a few days to weeks. This delay was caused by an interruption in network connectivity, increased caseloads, and staff turnover:
‘When it is the rainy days and the cases increase, sometimes it’s only two of us capturing data. Therefore, we would be late to capture… Late for a few days to one or two weeks.’ [Data clerk, FGD01].
In the first two quarters, there would be significant missing details on the notification forms. For instance, not all forms had barcodes, coordinates or patients’ addresses. Data clerks would identify patterns of missing barcodes and help identify whether facilities were underperforming or ran out of barcodes. This would be communicated to the case investigation supervisors. Supervisory staff reported that it is still common to find paper notification forms, mRDTs or DBS without barcode stickers due to forgetfulness, increased workload or reluctance from the clinic staff. Unidentifiable sample forms without barcodes would get discarded as no information was provided for identification or linkage. Quoting one of the participants in an FGD:
‘Yes, that are missing, maybe the facility code or anything like that, then maybe three or four times we have experience that the RDT [referring to mRDT] came alone like just the RDT without the filter paper, without a barcode. So, most of the time we just discard because there is nothing we can do or go back and find this person and do the thing again’ [Surveillance supervisor, FGD02]
This resulted in low linkability of samples because of inadequately barcoded samples, transcription errors and slow data-capturing processes. However, in the longer term, due to supportive supervision and re-training, cases linked at the household level increased with an increase in willingness and uptake of the programme.[12]
4.5 Process V: Analysis and reporting
Routinely, the NMP collates and analysis malaria data at the provincial level each month and shares the reports with the healthcare facility, district, provincial and national teams. For the integration of molecular resistance markers into the routine malaria surveillance system, the additional analyses included malaria case distribution and linkage of individual notifications to their molecular resistance markers. The NMP information officers and molecular laboratory team downloaded data from the DHIS2 each month and shared it with the research team for further analysis and reporting 1) maps of the distribution of malaria cases; 2) linkage of residential locality and health facilities with molecular resistance markers and 3) data quality. If any gaps were identified, activities were formulated for further enhancing resistance surveillance.
A total of 4,787 malaria cases were notified and 78.5% (n = 3,758) of cases were investigated in the study period. However, data wouldn’t always match, and since this data came from separate files within the DHIS2 system, it needed to be merged outside the DHIS2 environment, leading to extra workload for information officers with a risk of non-matching, duplication, or deletions of some cases, variables or values. Commenting about the workload and how the DHIS2 framework could lead to non-matching of data and hence linkage failure, an NMP officer said:
‘…I need to merge the NMC form together with the case investigation form and to take it outside the computer and merge; even if it is Excel, it is prone to changing formats and that could introduce issues that could lead to non-matching within the DHIS2 framework.’ [Information officer, KII03].
Fig. S2(a), (b) and (c) compare data from DHIS2 and the two lower levels (health facility and sub-district data capture centre); the notified case counts were only consistent in 40% (6/15) of the notifications and 20% (3/15) for cases investigated. Most of the matching evaluations occurred in the final two study quarters. Looking at the discrepancies, the two secondary levels (reports at the data collection centre vs data in the DHIS2) had less data variability compared to their primary source (health facility). Although the study’s quantitative analysis identified substantial duplication, the malaria team did data cleaning before sharing the datasets, as explained by one of the interviewees:
‘Once notifications were submitted and paper forms were captured in the DHIS2 system, ideally, all cases were merged and duplicates removed. In our analysis and feedback, some cases had duplicates that required further cleaning.’ [Information officer, KII03].
Analysis and reporting activities were part of routine practice before the integration of molecular resistance markers into the routine malaria surveillance system. However, due to the expansion of reported variables and the monthly downloads, merging and reporting required, this process became more complex and increased workload, especially in the first quarter. However, these extra activities were perceived as useful to the programme. The research team and the programme co-developed a training programme and data curation tools, engaging all involved in identifying gaps and improvements that further simplified the data analysis and reporting and improved the overall notification system.[20] This improvement reflected beyond the study and supported improvement for the programme reporting, quoting an IDI participant:
‘I think another thing that is good, our case investigation has also improved because I remember, when we started our baseline was 35% [for] 48 hours [time between case notification and investigation] and [for] 72 hours it was around 48%. So now on third quarter we reported above 65 (%) which was even above the target we set for this financial year. I think with them making the follow ups, having to go to the facilities to check on the stocks and everything has made them also be involved in going there and investigating the cases because the thing was XXXX and XXXX [SS4ME pilot programme staff names] would be analysing and supporting with feedback’ [Supervisory staff, KII02]
5. Discussion
This study identified factors affecting data quality, linkage, and consistency for integrating antimalarial resistance monitoring into routine malaria notification in a pre-elimination malaria setting in sub-Saharan Africa. Healthcare facilities, district and provincial level healthcare staff in Nkomazi Sub-district, Mpumalanga, South Africa were key beneficiaries, and their perspectives spearheaded this integration. Applying a process-oriented logic model, an iterative process for comprehensively analysing various factors that either facilitate or hinder successful integration was established, with end-to-end data flow and interaction with various users, beneficiaries, the existing systems and newly adopted technology. The programme’s overall performance was influenced by cross-cutting factors such as workload, training, perception, supervision, leadership and resources – acting as enablers or barriers based on their availability, adequacy and perception.
Increased workload was identified as a cross-cutting factor for all processes. Adequately trained staff who perceived the activity as useful maintained high performance levels, delivering better-quality outcomes even during months with a high malaria caseload, reduced staff and multiple reporting systems. In contrast, if staff did not perceive the activity as useful, data quality remained low, irrespective of resources and training provided. Previous studies have shown that a negative perception affects overall performance as well as the adoption and implementation of new technologies.[21,22] Low morale could lead to reluctance and the other activities being prioritised, contributing to poor data quality.
The impact of devices (GPS devices, tablets, barcode scanners) introduced or updated for the integration of molecular resistance markers into the routine malaria surveillance system depended on the healthcare workers’ skills and perception of the devices’ usefulness and ease of use. Optimal functionality of devices coupled with adequate training enhanced performance, leading to observable improvements in data quality, especially during and immediately post-training and supervision periods.[12] Conversely, malfunctions in devices, such as internet disruptions, barcode scanner failure to connect to the DHIS2, depleted GPS device batteries, or clinic-level shortages of barcode stickers, significantly impacted team performance, morale, and data quality. Other studies exploring technological adoption in the health sector highlight the crucial need to fully understand end-users’ needs and potential useful functionalities to facilitate usage and enhance quality. These studies stress the importance of user-friendly functionalities and overall device performance.[22,23]
This study found adequate supportive supervision and quality assurance tools to be influential in improving performance. This includes adequate financing, such as having long-term appointments without interruptions of contracts, a manageable workload and good collaboration among and between both government and non-governmental organisation staff. On the other hand, multiple parallel reporting systems, communication challenges, inadequate supervision, lack of quality control tools, unmanageable workload, and poor collaboration negatively impacted intervention implementation.
The study highlighted challenges at different operational levels, specifically in the malaria notification and case investigation processes. Although the NMC Notification Book served as the gold standard for reporting malaria cases, the survey results revealed parallel and occasionally interchangeable use of reporting systems. A 2019 review assessing the strengths and challenges of implementing DHIS2 across 11 Low- and Middle-Income Countries (LMICs) highlighted pervasive data quality issues, notably demonstrated in a Nigerian study where reported data in DHIS2 remained incomplete at least 40% of the time.[24,25] Despite encountering such data quality challenges in this study, more than half of the cases could be linked to resistance molecular marker data, with clear improvements in data quality observed over the study period.[12]
Contrasting responses were observed regarding follow-up phone calls for additional information —some healthcare workers felt it assisted with case investigations while others felt it strained the working relationships, negatively impacting staff morale and performance. This challenge relied primarily on staff perceptions and relationships. However, effective leadership, including the establishment of conducive communication channels, procedural guides, and efficient documentation systems, has successfully addressed similar challenges in comparable contexts.[23] This demonstrated the pivotal role of leadership and management in optimising communication and operations. Previous studies in similar settings have shown similar factors impacting the general adoption and implementation of information systems in health care.[21,26–29] A high level of perceived workload, data tools not being used as intended, and data quality issues were previously identified as challenges facing DHIS roll-out in Uganda and South Africa.[30,31]
Our study acknowledges some limitations in evaluation methodology, participant selection, and the analytical approach used for combining qualitative and quantitative data. No single framework was comprehensive enough for the evaluation of the integration of molecular resistance markers into a routine malaria notification system. Incorporating diverse technologies such as barcode stickers, filter papers, GPS protocols, and scanners introduced complexity to the evaluation process. The 2021 MRC framework for developing and evaluating complex interventions offers a comprehensive evaluation spanning a study’s lifecycle, from conception to implementation. However, it lacks detailed guidance on assessing individual components independently and primarily serves as a guiding framework without providing specific granularity for evaluation.[32] For the integration of molecular resistance markers into the routine malaria surveillance system, immediate end users are the healthcare staff and malaria case investigators. However, most determinant frameworks do not include end-users, and research has been scarce for improving the evaluation of how various end-users influence implementation effectiveness. Thus, our study employed an iterative process-oriented logic model.
Our results may be biased as only 42 staff members from the participating healthcare facilities consented to participate in the FGDs and IDIs, while 64 participated in the survey. The study results might not fully reflect the view of all staff in Nkomazi sub-district. However, the study team ensured diversity during participant recruitment to allow for different views and experiences. Most healthcare staff have multiple commitments, and their time is very limited. Hence, this might limit the study’s generalisability. More studies might be needed to establish the extent to which the results from this setting may be applicable in other pre-elimination contexts.
6. Conclusion and Recommendation
Overall, the factors influencing the integration of molecular resistance markers into the routine malaria surveillance system in this sub-Saharan pre-elimination setting were rooted in challenges inherent to the surveillance system itself, and were not specific to the integration of molecular resistance markers. This study underscores the intricate interplay of factors influencing malaria notification data quality and the integration of drug resistance markers within the routine surveillance system. It emphasizes the pivotal role of aligned perceptions, adequate resources, and supportive supervision in bolstering data quality, while also revealing the vulnerabilities stemming from device malfunctions, conflicting guidance, and disparate reporting systems. The evaluation frameworks used highlighted the need for more comprehensive models that consider the holistic healthcare environment, user perceptions, and nuanced interactions, particularly for complex interventions such as this. To enhance the implementation of such multifaceted interventions, future evaluations should include multiple countries to capture various healthcare settings, individual perceptions, and contextual nuances, enabling a more generalisable assessment and refinement of these interventions within routine healthcare systems.
Data Availability
All qualitative relevant data are within the manuscript and its Supporting Information files. The quantitative datasets generated and/or analysed during the study are available at the WWARN Tracking Resistance website (https://www.wwarn.org/tracking-resistance/artemisinin-molecular-surveyor).
https://www.wwarn.org/tracking-resistance/artemisinin-molecular-surveyor
8. Funding
The Smart Surveillance for Malaria Elimination Pilot in Mpumalanga, South Africa, was co-funded by the South African MRC and the Worldwide Antimalarial Resistance Network (WWARN). WWARN is funded by the Bill and Melinda Gates Foundation and the ExxonMobil Foundation. This research was also, in part, funded by the Wellcome Trust [Grant Number 220211]. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. Funders did not have any influence in the design of the study, data collection, analysis, interpretation of data and or writing the manuscript.
7. Acknowledgements
We would like to acknowledge the Mpumalanga Provincial and Nkomazi Sub-district Malaria Control Programmes of the National Department of Health of South Africa, Laboratory for Antimalarial Resistance Monitoring and Malaria Operational Research, Centre for Emerging Zoonotic and Parasitic Diseases, National Institute for Communicable Disease, Clinton Health Access Initiative, South Africa, Humana People to People (HPP), South Africa, all NMP personnel and healthcare facility staff who collaborated in this study for all their support.
9. References
