Development and evaluation of codelists for identifying marginalised groups in primary care

2.1. Step 1. Clearly define the clinical feature of interest a priori

Watson et al recommend beginning by clearly defining the clinical feature of interest [11]. To do this, they recommend using reliable sources of information, such as the National Institute for Health and Care Excellence (NICE) [12]. For the purposes of this study, we used the following definitions and sources to define marginalised groups of interest:

2.2. Step 2. Assemble list of codes that may be used to record the clinical feature

In the second step, an online thesaurus was used to develop a list of synonyms associated with the clinical features of interest. An overview of the synonyms used for each clinical feature is presented in Table 2.

We then searched for published codelists, relating to the clinical features, using the synonyms identified. One major resource used was the October 2023 Clinical Practice Research Datalink (CPRD) Bibliography [21]. CPRD is one of the largest research databases of electronic, anonymised, and longitudinal medical records from primary care in the world [22]. The CPRD Bibliography is a resource which provides a record of all published studies (>3,000 as of October 2023) using CPRD data. The bibliography aided in our search for relevant publications. The codelists used in these publications were subsequently downloaded, where they had been made available.

To capture codelists used in non-CPRD studies, we also searched for, and downloaded, relevant codelists from four widely used codelist repositories, namely: OpenCodelists [23], the Health Data Research UK [HDRUK] Phenotype Library [24], the University of Cambridge Codelist Repository [25] and the London School of Hygiene and Tropical Medicine Data Compass [26].

The codes from the codelists were then extracted and collated into separate excel files for ethnicity, LD, SMI and transgender. Duplicate codes were removed, and remaining codes grouped by the two main coding systems used in the UK: Read and SNOMED-CT. Read was generally used in primary care until 2018, and SNOMED-CT thereafter, following the enablement of SNOMED-CT within Primary care systems [27].

The aggregated codelists were then checked for accuracy using the CPRD code browser, which contains the diagnostic description, the alphanumeric Read / SNOMED-CT code originally used by the general practitioner (GP) to enter the clinical information, and CPRD’s proprietary ‘medcode’ / ‘medcodeid’ (which is simply a numeric equivalent of the Read / SNOMED-CT code) [27]. The tool allowed for easy browsing by either using a known code or description to check code accuracy. In line with the method described by Watson et al [11], inaccurate codes were removed or corrected, and additional codes (not previously identified) added if they were associated with the diagnostic definition of one or more eligible codes.

2.3. Step 3. Delphi review of codes

In the final step, the codelists were sent to a GP (IdVH) for review. They scored each code using the following three-point scale:

1. Definitely include: the code accurately defines the clinical feature of interest, and GPs would use it.

2. Uncertain: it remains unclear whether the code accurately reflects the clinical feature of interest, or whether GPs would use it.

3. Definitely exclude: the code does not define the clinical feature of interest, and GPs would avoid.

Codes assigned a score of 2 (‘Uncertain’) or 3 (‘Definitely exclude’) were reviewed by a second GP (BN). All codes assigned a score of 3 (‘Definitely exclude’) by both reviewers were removed from the codelists. All codes receiving a score of 2 from either reviewer were retained as ‘uncertainty variables’, which are recommended for sensitivity analyses [11]. These variables are typically less frequently used by GPs, but may identify additional, potentially relevant, cases [11].

2.4. Assessing codelist comprehensiveness

To explore the comprehensiveness of the developed codelists, we used the CPRD database browser to ascertain the number of people with at least one recording of an included code for each list. We restricted the population to those who had been registered with their CPRD Aurum practice since at least 1^st January 2019 and who had neither died nor deregistered from their practice by August 2022 (the date of last data collection).

In the context of CPRD studies, researchers have been able to acquire already categorised ethnicity data through linkage with the Hospital Episode Statistic (HES) [28]. Derived ethnicity data is also now available through CPRD at an additional cost [29]. The ethnicity codelist we generated will service to those researchers who are using other non-CPRD datasets, or ones who may not have access to either the linked HES data or the CPRD derived ethnicity.

For the other variables, validated codelists from other sources were selected as comparisons, seeking to match the scope of each list as closely as possible. Codelists for LD and SMI were sourced from the Quality and Outcomes Framework (QOF: the pay for performance scheme for general practices in the UK) [30]. QOF includes incentives for practices to keep a register of patients with certain conditions, including LD and SMI, and produces validated codelists for this purpose. We used lists from the QOF Business Rules version 45; “Learning Disability codes” (list name LD_COD) and “Psychosis and schizophrenia and bipolar affective disease codes” (list MH_COD). For the transgender category, no QOF list exists, and literature is slim. We identified two identical previously published codelists that were used to compare to our newly developed codelist.

4.1. Summary

We created comprehensive codelists for identifying ethnicity, LD, SMI and transgender records in EHRs. Compared with the commonly used alternatives, our codelists improved data capture by 76.6% for LD, 68.9% for SMI and 36.9% for transgender records. These codelists have been made publicly available for others to use in their own research: https://osf.io/8skze/. They are readily available for use in UK research, but need to be adapted for international research, where different code systems are used. We provide code descriptions, alongside codes, to help facilitate this process.

4.2. Strengths and limitations

This study has several strengths. First, code lists were identified through multiple sources, including CPRD publications and code list repositories, maximising the chances of identifying potentially relevant codes. Second, instead of accepting codes at face value, a Delphi review was implemented, with potentially irrelevant codes removed or segregated for sensitivity analysis.

This study also has a number of limitations. First, it only used codes from code lists arising from CPRD publications, or uploaded to code list repositories, and, despite several safety netting procedures, it is possible not all relevant codes were captured. This is exemplified by the fact that only 29% of relevant publications made their code lists publicly available, meaning there is potential for many codes to be missed. Second, while we compared the comprehensiveness of our codelists against commonly used alternatives (e.g. QOF), the comparisons were not like-for-like, due to differences in population definitions. For example, the QOF definition of SMI does not include severe major depression, and these differences may partially explain some of the increase observed in our codelist.

4.3. Comparison with existing literature

This research builds on the extant literature by synthesising the efforts of those who have previously curated code lists for the identification of marginalised groups. For example, Boyd et al [31] previously created a code list for identifying people who are transgender, which included a total of eight Read codes. Our list includes an additional 30 Read codes (along with the corresponding SNOMET-CT codes), enabling higher resolution and identification of people who are transgender.

4.4. Implications for research and / or practice

The findings of our study have several implications for research. First, we found that only a small proportion of researchers made their codelists publicly available. Efforts are therefore needed to encourage researchers to share their codelists. This will expedite inequalities research and prevent duplication of work. Second, while the percentages of ethnicity found in the population closely mirrored the frequencies reported in the UK census 2021 [15], we found that other marginalised groups were under-represented in the CPRD dataset (1.00%, 1.18% and 0.02% were recorded as having a LD, SMI, or transgender code, respectively, compared with national estimates of 2.16%, 2.00-3.00% and 0.20% [32, 33, 34]. Further research is needed to understand why these frequencies do not match national estimates. Finally, there are many more groups experiencing inequalities, for whom we have not developed codelists, including other LGBT groups and long-term conditions. We encourage others to follow our methods to develop comprehensive codelists for the identification of these groups.

The outputs from our study also have implications for policy. For example, the code lists can be used by clinicians for audits and service improvement projects, to ensure correct identification and classification of individuals, and improve the reliability and validity of findings.