Abstract
Vector-borne diseases are an increasing threat to human health and well-being in the United States. Understanding public perception and practices taken to reduce vector abundance and vector-human contact can guide effective interventions that can mitigate health risks. Nevertheless, vector-borne disease surveys, which are widely used in the field to understand public perception and practices, are often inconsistent in terms of structure and implementation. This protocol is designed to provide guidance for public health professionals and researchers in the development of future KAP studies by ensuring uniformity in design and structure. This manuscript describes a rigorous three-phase protocol for the development of standardized vector-borne disease survey modules that can be used throughout the United States to generate data that are comparable across diverse regions. During phase one, a workshop with subject matter experts and a comprehensive literature review will be conducted to identify survey domains and generate items of interest. Survey items will also be mapped based on two theoretical frameworks, the Health Belief Model and the Risks, Attitudes, Norms, Abilities and Self-Regulation framework, to help gauge individual perceptions of risk severity and self-efficacy. During phase two, a group of expert judges will evaluate survey items based on content relevance, representativeness, and technical quality. During the final phase, cognitive interviews and surveys with target audience groups will be conducted to ensure the face validity of the modules. Participants will be drawn from a diverse range of educational and geographic locations, and computer-assisted personal interviewing will be implemented. The surveys developed through this protocol will be available to researchers aiming to gain deeper insights into the public’s knowledge, attitudes, and practices concerning vector-borne diseases, facilitating the collection of more comparable data across various regions in the United States.
Introduction
Vector-borne diseases (VBDs) are an increasing threat to human health and well-being in the United States (US). Over the past two decades, transmission of tick-borne diseases has doubled, while outbreaks of mosquito-borne diseases persist across the continental US and its territories. This increase is associated with shifting seasonal patterns of transmission and the wider spread of disease vectors and their pathogens [1, 2]. Engaging the public in prevention and control measures is crucial to mitigating these risks. Understanding public perception and practices taken to reduce vector abundance and vector-human contact can guide effective interventions.
Knowledge, attitudes, and practices (KAP) surveys are one of the most common assessment tools used to understand public health issues in a community [3]. These surveys have been widely used within the context of VBDs. Knowledge questions seek to understand respondents’ level of understanding about a VBD topic [3, 4]. This could include questions related to knowledge of vectors and their presence, disease transmission and symptoms, and prevention strategies including vector control and personal protection. Attitude questions relate to the emotional, motivational, perceptive, and cognitive beliefs that influence the behaviors of an individual [4]. For VBDs, these questions could pertain to people’s attitudes about recommended practices and guidelines regarding the disease, and their perception of risk and disease severity. Practice questions ask about the current actions of an individual, such as the measures undertaken to reduce VBD risk at individual, household, and community levels [4]. In general, KAP surveys are easy to design, quick to implement, easy to interpret, produce quantifiable data, and often allow for generalizations of small sample results to larger populations [5]. Ideally, these surveys use an underlying theoretical framework of behavioral theory, such as the Health
Belief Model or the Risk, Attitudes, Norms, Abilities, and Self-Regulation framework. These underlying constructs are important to understand where efforts must be focused to enhance the adoption of prevention and control methods [6, 7]. These studies set baseline values for subsequent evaluations, evaluate the efficacy of health education initiatives, and inform localized and culturally contextualized intervention approaches [8].
This study aims to create standardized modular surveys for the general public, enabling data comparison across various geographical regions and projects undertaken by public health professionals and researchers in the US. Three phases will be implemented to achieve this goal: 1) identifying survey domains and items; 2) developing modules; and 3) evaluating modules.
This manuscript outlines the detailed protocol to create these modular surveys, which will cover the most prevalent VBDs found in the US as follows:
General mosquito-borne disease module
Specific mosquito-borne disease modules
Dengue
Zika
West Nile
St. Louis Encephalitis
General tick-borne disease module
Specific tick-borne disease modules
Lyme disease
Rocky Mountain Spotted Fever
Methods
Study Design
Modular survey development is based on the knowledge, attitudes, and practices (KAP) model [3, 4]. The development process consists of three phases, following the methodologies delineated by Hinkin and Boateng (Fig1) [9, 10]. Ethical approval was obtained through the University of Arizona Institutional Review Board in July 2023.
An overview of the three protocol phases.
Phase One: Domain Identification and Item Generation
In this initial phase, a workshop with subject matter experts and a comprehensive literature review will be conducted to identify survey items used in previous studies. The workshop and literature review will be implemented to identify domains and generate items of interest.
Solicitation of Expert Panel for Key Domains
An initial workshop will be conducted to identify primary domains of knowledge, attitudes, and practices that are considered necessary for the control and prevention of mosquito-borne and tick-borne diseases. Approximately 20-30 experts will be drawn from applied public health, vector control agencies, and academic researchers conducting applied research related to VBDs. An asynchronous method, Idea Flip, will be used to solicit initial key domains. A follow-up workshop will bring participants together to identify additional themes and prioritize themes for tick-borne and mosquito-borne diseases.
Literature Review and Assessment of Existing Scales
The goal of the literature review is to identify and analyze existing surveys and develop a set of domains and items to be used throughout the modular surveys. This is the first step in developing sound measures and ensuring content validity [9]. The literature review will be conducted in several stages. Initially, a scoping review will be undertaken using specific search strategies. Search criteria will include “KAP,” “KAB,” or “KAPP” combined with “AND” followed by specific mosquito and tick-borne diseases using Boolean searching (S1 Appendix).
This review will assess current practices regarding the utilization of KAP surveys, the themes addressed, and the way questions are formulated. The research team will contact the authors to obtain survey tools when KAP surveys are not readily accessible within the manuscript. The following data will be retrieved from the KAP surveys using a data extraction form: the first author’s name, publication year and title, diseases, vectors, region and country, study design and sample size, how the survey/tool/scale was developed, and sources referenced, how the survey/tool/scale was validated, and the number of items included. Further solicitation of survey tools will be made from applied public health departments using personal networks to identify potentially unpublished tools. Items will be extracted and included in a master database that will be used to determine potential items for the draft modules.
Subsequently, public-facing materials such as guidelines, posters, and fact sheets will be collected from selected state health departments and the Centers for Disease Control and Prevention and World Health Organization websites. The selection of state health departments will be based on the severity of the mosquito-borne and tick-borne diseases based on the incidence rates and reported cases in the health department jurisdiction. Relevant data will be collected according to a predefined data extraction format, which includes reference, source, format (e.g., poster, fact sheet), vector, disease, domain (Knowledge, Attitude, or Practice), and sub-domain (e.g., symptom, transmission).
Development of the draft survey tools
The data systematically collected from the scoping review of peer-reviewed papers on KAP related to VBDs, along with prevention and control guidelines, will serve as the foundation for developing the modules.
Underlying theoretical frameworks
As the goal of KAP surveys for VBD prevention and control is to identify areas to focus efforts that motivate action, survey items will be mapped to the components of two theoretical frameworks for health behavior, the Health Belief Model (HBM) and the Risks, Attitudes, Norms, Abilities and Self-Regulation framework (RANAS) [6, 7]. Employing the HBM as a theoretical framework in survey development can be beneficial. This model helps gauge individual perceptions of risk severity as well as self-efficacy, or an individual’s belief in their ability to take meaningful action to mitigate risk [6]. Given that risk perception is intrinsically aligned with motivating behavior change, these elements are essential in gauging the likelihood of success of an intervention. The RANAS framework complements the HBM by addressing societal norms, an aspect absent in the HBM, thus emphasizing the importance of tailoring and contextualizing specific behaviors/behavior changes within a population [7].
Phase Two: Content Validity through the Delphi
During phase two, content validity will be measured through the Delphi process. A group of expert judges will evaluate each item for content relevance, representativeness, and technical quality [10]. These expert judges will be comprised of representatives from multiple groups including:
Vector control personnel
Health department personnel
Academic VBD researchers
Expert opinions will be solicited via email. The initial recruitment will be among 30-50 individuals with an anticipated response of 70%. Those who agree to participate will engage in two rounds as part of the Delphi process. Strategies to keep participants engaged will be employed, including personalized survey invitations and periodic reminders. Three follow-up reminders will be sent every two weeks following the initial distribution. One phone call reminder will be conducted as needed for the second round to ensure a 70% follow-up for round two.
During round one, the expert panel will receive a link to a web-based Delphi questionnaire, which will include a consent form, background information, instructions for the survey, a short demographic survey, and the modules. A content validity ratio (CVR) will be employed as a measurement consensus for items. Each item will be scored using a 4-point scale: 4=Highly Relevant, 3= Highly Relevant But Needs Rewording, 2=Somewhat Relevant, 1=Not Relevant, and the CVR will be calculated [11]. The members of the panel will be able to suggest rephrasing, provide rationales for their choices through a comment box attached to each item, and suggest missing or new items. This study will involve at least 20 experts, so a minimum CVR value of 0.42 will be used [12]. Items that fall below this threshold will be removed.
Responses from round one will be analyzed and summarized into one feedback report, which will include summary statistics including the standard deviation and means for each item, the level of consensus, coefficient variation, participants’ feedback and comments, and the decision regarding item selection based on predefined consensus level using the CVR. The feedback report will be sent back to the expert panel with the introductory material for the second round. During round two, experts will be asked to review the feedback report and re-evaluate items.
Once a consensus has been met for all items, results will be sent out to the panel.
Phase Three: Face Validity through Cognitive Interviews and Survey
During phase three, cognitive interviews and surveys with target audience groups will be conducted to ensure the face validity of the modules. End users will judge whether the items are appropriate to the targeted construct and assessment objectives [10]. Given the target of the survey modules are adults who reside in areas with mosquito-borne disease and tick-borne disease activity, participants will be drawn from a diverse range of educational backgrounds and geographic locations (i.e. urban, suburban, and rural).
Cognitive Interviews
To assess the extent to which questions reflect the domain of interest and answers produce valid measurements, we will conduct cognitive interviews with the target population [13]. The cognitive interview process will use think-aloud and verbal probes to identify how participants process and respond to items within the surveys [13]. There will be two groups of interviewees: one group will review all mosquito modules (n=20), and one group will review all tick modules (n=20). With the final target of the public residing in areas with mosquito and tick-borne disease risk in mind, interviewees will be chosen to represent urban and rural areas and varying educational levels. Interviewees with experience conducting qualitative research will ask each survey question to the participant. After the interviewee answers the question, probing questions will be asked to obtain feedback and allow participants to verbalize their thought processes. Example probes will include, “can you explain what [key term] means in your own words?”, “I noticed you hesitated, can you tell me more?”, “did you find this question easy to answer?” and overall questions, “did you find any of these questions difficult to answer?”, “which questions were hard to answer”. Items will be modified based on the feedback for improved clarity [10].
Survey Administration
Computer-assisted personal interviewing (CAPI) using an online survey tool will be conducted to ensure that the surveys can collect data with minimum measurement errors and discriminate between audiences who are knowledgeable and have good practices versus those who may not [10]. In-person interviews will also be conducted (n=50) to delineate differences in how questions should be asked during in-person interviews when online tools are not used. Scale items will be evaluated with three sample groups that reflect and capture the range of the target population. These groups are individuals involved in the control and prevention of VBDs (Group 1, n=75), public health personnel not affiliated with VBD programs (Group 2, n=75), and the general public (Group 3, n=800). English-speaking adults 18 and older will be eligible to participate, excluding incarcerated individuals. It is anticipated that this will represent three levels of knowledge, attitudes, and practices around VBDs, with Group 1 exhibiting the highest level of knowledge and engagement in prevention activities, Group 2 having a moderate level of knowledge and participation, and Group 3 having the least and most variable knowledge and participation. Participants for Group 1 will be identified by extending invitations through collaborative networks such as the American Mosquito Control Association, Centers for Disease Control and Prevention Centers of Excellence for Vector-borne Diseases and Regional Training and Evaluation Centers (TECs) and other known networks. Group 2 will be recruited through public health organizations such as National Association of County and City Health Officials (NACCHO), current author connections with public health agencies. Snowball sampling will be conducted for both Group 1 and 2, whereby individuals can refer others who should take the survey at the end of their survey. The investigative team will then invite those individuals. Group 3 will be recruited using a public survey panel. As one of the goals is to assess how knowledge, attitudes, and practices differ based on age, gender, residential area, and occupation recruitment, goals will be stratified by these variables.
Sample sizes per group will be conservatively determined based on comparisons between Group 1 and Group 2 given these are more specialized populations with a much smaller target population than the general population. We estimate that a meaningful difference in proportions is 20% or more between the vector control population and those in general public health practice. Sample size estimates for a one-sided difference in proportion (20% vs. 40%) with standard (α = 0.05, β = 0.80) are 64 per group; this increases to 74 per group (30% vs. 50%). A wide range of differences in proportion is anticipated given the survey will intentionally include more common and more rare knowledge and practice items to facilitate discrimination between populations. The higher sample size in Group 3 (general public) will allow internal comparisons among demographic sub-groups including individuals from differing education and income levels.
Analysis Plan
Descriptive statistics will be computed for participant demographics including age, gender, educational attainment, and residential setting (urban or rural). The levels of knowledge, attitudes, and practices across demographic variables such as age groups, gender, occupation, and education levels will be analyzed to identify vulnerable populations with limited understanding and suboptimal practices. This analysis aims to detect statistically significant differences among these groups (Groups 1, 2, and 3). Further analysis will be conducted as needed including Cronbach’s alpha to assess the reliability and factor analysis to refine the survey by removing items with low loadings.
Discussion
This manuscript describes a protocol for the development of standardized VBD survey modules that can be used throughout the US to generate data that are comparable across diverse regions. Based on our literature review, VBD KAP surveys are widely used in the field, but they are inconsistent in terms of structure and implementation. Two of the most pressing issues were inconsistencies with the structure and wording of questions, and the reliability and validity of questions. Among the reviewed surveys, the classification of knowledge, attitudes and practices questions was not uniformly applied across all studies. For example, items listed in the practices section of one survey were listed in the attitudes section of another survey, and vice versa. There were also instances when different research teams were trying to obtain the same information, but because their survey questions were asked differently, they were not comparable. Key areas in which questions differed included:
Use of dichotomous vs. Likert scales for frequency or intensity of actions, perceptions and/or knowledge
Time scales of reference (seasonal vs. past week vs. past year, vs. ever)
Inclusion or exclusion of additional context in inquiring about actions (i.e. no context vs. when you are outside vs. when you are being bitten by mosquitoes)
Differences in pre-text – provision of additional information about the next questions
Order of questioning, though many proceeded from knowledge to attitudes, to practices
Use of a behavioral theory as an underlying theoretical framework
Furthermore, it was often unclear how research teams developed their surveys because references, theoretical frameworks, and sources for question derivation were not provided. Findings differed based on how questions were asked and the answer choices that were allocated, impeding the comparability of results across different surveys.
This protocol is designed to provide guidance for public health professionals and researchers in the development of future KAP studies. By ensuring uniformity in KAP survey design and structure across geographical regions, the current lack of comparability can be addressed. Establishing a three-phase process that incorporates a comprehensive literature review, a rigorous process for content validation, and testing for face validity, modular surveys can be designed for broader dissemination and use. This is particularly important in areas where vectors and/or pathogens are newly emerging so that patterns of disease transmission can be compared across geographic regions. This will aid in the development and implementation of practical and sustainable interventions to reduce the burden of disease. The surveys developed via this protocol will be available for use by research teams that are interested in better understanding the knowledge, attitudes, and practices of the general public regarding VBDs. This protocol process does not include survey development for Spanish-speaking populations. However, after the initial modules have been finalized, future development will expand to include these populations.
Data Availability
NA
NA
Acknowledgments
We wish to thank Joshua Arnbrister for his support during the early stages of this project.
S1 Appendix Supplementary Table
