Developing a Discrete Choice Experiment to Understand Patient Preferences in Resource-Limited Settings: a Six-Step Guide

Step 1: Formative Work

We began by consulting the literature and conducting qualitative research. The literature review focused on previous applications of DCEs in low- and middle-income countries (LMIC). We searched PubMed with the terms “discrete choice” and “low- and middle-income countries.” This search yielded 56 results which directed the qualitative interview guide. For instance, Zanolini et al. found that provider attitude is a significant driver of preferences in patients with HIV in Zambia.¹² To evaluate whether provider attitude also influences patients with non-communicable diseases (NCDs) in rural Uganda, we included questions in our in-depth interview guide about relationships with healthcare providers. This example illustrates how the literature review guided our participant interviews.

We interviewed 18 participants in Nakaseke District, Uganda with hypertension, diabetes, or both about their experiences accessing healthcare and medicines for these conditions. We chose this sample size as it would likely achieve thematic saturation.¹³ Participants were asked about 3 themes: accessing care, accessing medicines, and relationships with healthcare providers. Interviews were conducted in Luganda by a single research assistant, then transcribed and translated into English. A directed coding approach was used to code the transcripts and identify attributes and their appropriate levels to be included in the DCE. Directed coding analysis is performed when there is a pre-existing direction about what the researcher is looking for in the qualitative data.¹⁴ This approach allowed 3 qualitative coders, including 2 Ugandan coders, to identify concepts in the data that could be attributes in the DCE. Consensus was reached between coders through an iterative coding process and continuous discussion to ensure agreement on code assignments and definitions. 10 tentative attributes emerged: the presence of peer support groups, health education, interactions with healthcare providers, range of treatments, perceived quality of care, waiting time, use of herbal medicine, getting to the facility, costs of treatment, and availability of medicines.

Step 2: Attribute Selection

It is important to limit the number of attributes to fewer than 10 to minimize participant fatigue during the completion of the task.⁹ DCE tools that present too significant a cognitive burden are likely to lead to respondents using a simplifying heuristic to make decisions, such as selecting based on a single attribute.¹¹ Previous DCEs administered in LMICs incorporated 5 or 6 attributes, which led us to select 6 attributes.^12,15,16 To determine which of the 10 tentative attributes would be included in the DCE, a multi-disciplinary team of experts in public health, medicine, anthropology, and NCDs from Makerere University, Yale University, and New York University gathered to discuss the tentative attributes. The Nominal Group Technique was used to reach a consensus on which attributes would be included in the final DCE tool.¹⁷ Using this well-established method, the experts were presented the tentative attributes, discussed the pros and cons of including each in the DCE, and rank-ordered each attribute. The rank order was discussed and a final re-ranking was performed. Primary reasons for the inclusion of attributes were amenability to intervention development and impact on patients’ experience of their care based on interview results. The team reached consensus on a list of 6 attributes: presence of peer support groups, education at the facility, interactions with healthcare providers, getting to the facility, costs of treatment, and availability of medicines. We excluded the use of herbal medicines, range of treatments, perceived quality of care, and waiting time because they were felt to be represented by 1 of the other 6 priority attributes, unamenable to intervention or ambiguous in interpretation based on the interview data.

Step 3: Attribute Level Selection

To determine levels for each attribute, the team considered the range of situations encountered by patients as described in the in-depth interviews. In the attribute levels, we sought to reflect the realistic range of situations that patients currently encounter.⁹ For instance, the “costs of treatment” attribute presents a range of monthly costs, between zero and 20,000 Ugandan Shillings (approximately 5 USD). Free treatment represents patients attending the public district hospital for appointments and medicines. Each of the other attribute levels presented reflects the reported out of pocket costs when patients attend private clinics and/or pharmacies. To ensure that we presented a realistic range of attribute levels for each attribute, we reviewed the ranges as reported in interviews and verified these with members of the team who are healthcare workers in the district where the research was based. Researchers might choose to include potential attribute levels that describe aspirational, rather than currently realistic, healthcare options. This was the case for our attribute level “I receive more than a month’s worth of medicine from the facility pharmacy” within the “availability of medicines for my condition” attribute. Though not widely offered, dispensing of greater than 1 month’s supply of medicines is an approach increasingly used in differentiated service delivery for HIV care in LMICs.¹⁸ While this option is not currently available for patients with NCDs in Uganda, we included it as an attribute level because of the research team’s interest in future expansion of differentiated service delivery models into NCD care in Uganda. The final attributes and attribute levels we determined based on this process are shown in Figure 3. Consistent with the Conjoint Analysis Good Practice Checklist, we avoided presenting attribute levels as ranges. Ranges result in variability in how respondents interpret each level, thereby limiting analysis. Additionally, we limited each attribute to a maximum of 4 attribute levels in accordance with good practice.¹¹

• Download figure
• Open in new tab

Figure 3:

Attribute and their respective levels included in the discrete choice experiment. (1 USD is approximately equal to 3700 Ugandan Shillings)

Determining how patients think about a given attribute was an important consideration in our discussion of attributes. We aimed to define each attribute level using terms that patients used in the interviews. In particular, the attribute “getting to the facility” was challenging to frame due to the diverse ways in which patients described travel in the context of seeking healthcare. Patients typically described either the cost of transport or the length of the journey as the challenge in getting to the healthcare facility. In an attempt to capture both types of descriptions, we created attribute levels reflecting both distance and cost. For instance, traveling 5 km by motorcycle typically costs about 5,000 Ugandan Shillings, so these were combined into an attribute level. We recognize this approach has drawbacks as we could not parse out whether participants’ selections were based on distance or cost.

Step 4: Selection of DCE Design

Health researchers who aim to design a DCE face a series of decisions in selecting the most appropriate design for their purpose. First, researchers must select between a full and a fractional factorial design. A full factorial design includes choice sets of all possible attribute level combinations for participants to choose between. Conversely, a fractional factorial DCE design presents only a subset of the possible combinations.² Most DCEs, including ours, employ a fractional factorial design as the total number of possible combinations is too many to present to respondents.¹²

Second, we considered how many choice sets to present to each respondent. This DCE required respondents to consider 6 attributes in their selection of an ideal health facility for NCD care. The cognitive burden of this decision-making task could be compounded by variable health literacy levels in rural Uganda. The literature suggests that presenting between 8 and 16 choice sets is typically appropriate, but that reducing the number of choice sets also reduces the cognitive burden of the decision-making task.^9,11 Following a discussion by the research team, we decided to present 8 choice sets to each participant to minimize the cognitive burden of the task. Additionally, as we did, pictorial representations of each attribute can be used to support the cognitive process of participants, especially in low literacy populations.

Third, in order to maximize the number of attribute level combinations included in the DCE, we created 3 different versions. Creating multiple versions of the tool is a strategy to increase the number of combinations presented to respondents, thereby increasing the coverage of hypothetical health scenarios evaluated.¹² This resulted in a total of 24 choice sets, with each participant only completing 8 choice sets. When deploying the DCE, we block randomized distribution of the survey versions by study site to ensure adequate representation from each study site across all 24 choice sets.

Fourth, one must consider whether to create a DCE with a full or partial profile.¹¹ A full profile design describes a DCE in which the attribute level presented for each attribute varies with every choice set presented.¹⁹ A partial profile describes a DCE in which certain attribute levels are omitted between choice sets and are used to handle DCEs with many attributes.¹⁹ In partial profile DCEs, respondents are instructed to assume that any omitted attributes are the same between the 2 options. However, we cannot know the attribute level that a respondent assumes for an omitted attribute, thereby limiting the ability of the DCE to reveal interactions between attributes.¹⁹ Since partial profile designs yield insufficient information to fully assess statistical interactions among attributes, it is preferable to use a full profile if the DCE has few attributes.

Finally, one must decide whether to include an opt-out option, in which participants are allowed to choose neither of the presented scenarios. The alternative is a forced decision design.¹¹ While presenting respondents with an opt-out choice more realistically reflects real-world circumstances, it can limit data interpretation because it does not reveal any information about the preference between 2 options.¹¹ It has also been demonstrated that participants with lower educational levels are more likely to opt-out of a decision task.²⁰ A dual response design consists of choice sets in which the participant is forced to make a decision, then for that same choice set asked if they would prefer to opt-out. This design provides an efficient means to minimize the learning effect, which describes how participants are less likely to opt-out for choice sets that are presented later on in the choice experiment.²⁰ However, this design does increase the complexity and time it takes participants to complete the task. Given the cognitively demanding nature of DCEs and our concern about a high level of non-response due to the limited education level of our target population, we used a forced decision design. Our final DCE consisted of a fractional factorial, full-profile forced decision design that presented 8 choice sets to each respondent.

Step 5: Determination of Attribute Level Combinations

The next step was optimizing the combination of attribute levels to improve the quality of the DCE output. We employed a Bayesian approach which takes into consideration common-sense assumptions about preferences within each attribute.²¹ For all variables, we assumed that the least burden on the patient would be the preferred option. For instance, for “costs of treatment” we assumed that paying less would be preferred, such that zero Uganda Shillings and 20,000 Ugandan Shillings (approximately 5 USD) monthly would be the most and least preferred options, respectively. This approach ensures that there are no uninformed choice sets presented and avoids creating a choice set in which a scenario is preferable in terms of all attributes.²¹ We used JMP software (Version 14) because we found it to have a user-friendly interface, an extensive user guide, and online user forum.²² JMP creates a Bayesian D-optimal DCE design in the multinomial logit model. D-optimality, in this case, means that the algorithm begins with a random starting design and over the course of the user’s determined number of random starts, selects the best design.²¹ Optimizing the combination of attribute levels helps create a DCE that yields more informative results.

Step 6: Enhancement and Assessment of Tool Comprehensibility

In step 6, we planned how to increase the comprehensibility of the tool upon its administration. We identified 3 domains that impact the comprehensibility of the tool: the introduction to, delivery of, and appearance of the instrument. In the written introduction to the tool displayed before the decision-making task, we provided an explanation of the task and a description of each of the attributes. Then, we planned for the tool to be delivered on tablets in the pre-testing and administration phases. Tablets aid understanding of the tool by allowing respondents to visually follow along with the research assistant’s description of the choice set.¹² As research assistants led participants through the task, they would describe each option within a choice set to participants and point on the tablet to the attribute level they were describing. We selected KoBoToolbox, a freely available data collection application suited for rural settings because it allows for offline data collection. This is an important feature in settings with variable internet access. Finally, we aimed to make the appearance of each choice set easy to understand and follow. Choice sets were created in a tabular format to clearly depict the attributes and their corresponding attribute levels for each facility. We also incorporated images representing each attribute to remind respondents of the definition of that attribute (Figure 3). Images are also thought to help reduce the burden of the decision-making task by making it easier to summarize each presented option.¹² These images were designed by Makerere Medical Illustrations after consultation with the lead author. The team of researchers then reviewed these images to ensure that they accurately represented each attribute. Self-administration of the DCE is an alternative administration approach researchers can consider. To support this approach in a low literacy population, a voice recording would need to be matched to the instructions, each attribute, and each attribute level for each unique choice set. Self-administration might be preferable in situations wherein research staff numbers are limited relative to the number of required participants.

Lastly, we pre-tested the tool by administering it to 7 randomly selected respondents from our target population.⁹ Pre-testing participants were presented 8 choice sets and asked to freely express their rationale for making each selection. This feedback revealed that respondents understood the tasks being asked of them and could complete all choice sets without significant cognitive burden. If the reported cognitive burden was too high during pre-testing, the tool would have required alteration, such as reducing the number of attributes or choice sets.

We have presented a detailed process guide for health researchers to reference in developing rational, locally-relevant DCEs for RLS. This guide describes the challenges we faced and decisions we made in the development of our DCE. We hope this guide will help increase the global use of DCEs as an informative tool to prioritize intervention design in RLS.