OpenSAFELY: Representativeness of Electronic Health Record platform OpenSAFELY-TPP data compared to the population of England ============================================================================================================================= * Colm D Andrews * Anna Schultze * Helen J Curtis * William J Hulme * John Tazare * Stephen JW Evans * Amir Mehrkhar * Seb Bacon * George Hickman * Chris Bates * John Parry * Frank Hester * Sam Harper * Jonathan Cockburn * David Evans * Tom Ward * Simon Davey * Peter Inglesby * Ben Goldacre * Brian MacKenna * Laurie Tomlinson * Alex J Walker ## Abstract **Background** Since its inception in March 2020, data from the OpenSAFELY-TPP electronic health record platform has been used for more than 50 studies relating to the global COVID-19 emergency. OpenSAFELY-TPP data is derived from practices in England using SystmOne software, and has been used for the majority of these studies. We set out to investigate the representativeness of OpenSAFELY-TPP data by comparing it to national population estimates. **Methods** With the approval of NHS England, we describe the age, sex, Index of Multiple Deprivation and ethnicity of the OpenSAFELY-TPP population compared to national estimates from the Office for National Statistics. The five leading causes of death occurring between the 1st January 2020 and the 31st December 2020 were also compared to deaths registered in England during the same period. **Results** Despite regional variations, TPP is largely representative of the general population of England in terms of IMD (all within 1.1 percentage points), age, sex (within 0.1 percentage points), ethnicity and causes of death. The proportion of the five leading causes of death is broadly similar to those reported by ONS (all within 1 percentage point). **Conclusions** Data made available via OpenSAFELY-TPP is broadly representative of the English population. **Summary** Users of OpenSAFELY must consider the issues of representativeness, generalisability and external validity associated with using TPP data for health research. Although the coverage of TPP practices varies regionally across England, TPP registered patients are generally representative of the English population as a whole in terms of key demographic characteristics. **Key messages** * There is regional variability across England in terms of key population characteristics * Users of OpenSAFELY should carefully consider the issues of representativeness, generalisability and external validity associated with using TPP data for health research. * TPP registered patients are a representative sub-sample of the English population as a whole in terms of age, sex, IMD and ethnicity. * The proportions of the five leading causes of death in TPP in 2020 are broadly similar to those reported by ONS. Keywords * OpenSafely * Representative ## Background OpenSAFELY is a secure health analytics platform created by our team on behalf of NHS England. OpenSAFELY provides a secure software interface allowing analysis of pseudonymised primary care patient records from England in near real-time within highly secure data environments. OpenSAFELY software is currently deployed within the secure data centres of the two largest electronic health record providers in the NHS: EMIS and TPP, and is delivering federated analytics where the same data curation and analysis code executes in each environment. To date more than 20 publications have used the OpenSAFELY platform, focused on delivering vital and urgent results related to the global COVID-19 emergency. Some of these papers have used the federated analytics functionality more latterly available in OpenSAFELY to deliver combined analyses across 58 million patients’ data in both OpenSAFELY-EMIS and OpenSAFELY-TPP (1–5); however the majority of analyses published during the pandemic specifically used OpenSAFELY-TPP which covers 40% of general practices in England, those using SystmOne Electronic Health Record (EHR) software produced by TPP. The use of data from EHR providers is an invaluable tool for health research, however data is primarily collected for clinical use and not specifically with research in mind. As these datasets are not a random sample of the population of interest, it is important to understand the representativeness of the data. The deployment of TPP SystmOne software is known to be geographically clustered (6), and factors such as sex, age, ethnicity and levels of deprivation, which are important clinical risk factors for death from COVID-19 (7), show regional variability across England (8). Key outcomes such as causes of death also vary by region (9). However, little is currently known about how the characteristics of patients in TPP practices compare to the population at large. In order to aid the interpretation of ongoing COVID-19 research projects in OpenSAFELY-TPP we therefore set out to compare key demographic characteristics of patients registered with TPP practices to national estimates from the Office for National Statistics (ONS). We also compared the distribution of the five leading causes of death registered in ONS between 1st January 2020 and 31st December 2020 to deaths registered in TPP during the same period. ## Methods ### Study design #### Data Source Primary care records managed by the GP software provider TPP were linked to ONS death data through OpenSAFELY-TPP, a data analytics platform created by our team on behalf of NHS England to address urgent COVID-19 research questions ([https://opensafely.org](https://opensafely.org)). Similarly pseudonymized datasets from other data providers are securely provided to the EHR vendor and linked to the primary care data. The dataset analysed within OpenSAFELY-TPP is based on 24 million people currently registered with GP surgeries using TPP SystmOne software. **It includes pseudonymized data such as coded diagnoses, medications and physiological parameters. No free text data are included**. Further details on our information governance can be found on page 20, under information governance and ethics. The UK Census collects individual and household-level demographic data every 10 years for the whole UK population. Data on ethnicity were obtained from the 2011 UK Census for England. In addition to census data, ONS release annual mid-year estimates of the resident population of England produced using a cohort component method (10). Data on IMD, Age and sex were obtained from the 2020-mid year estimates and estimates of the 5 most common causes of death in 2020 were obtained from ONS mortality statistics published via NOMIS (11). #### Study population For demography and coverage analyses, patients were included in the study if they were registered at an English general practice using a TPP SystmOne clinical information system on 30th June 2020. For analysis of causes of death, patients were included if they were registered with an English general practice using a TPP SystmOne clinical information system on the day of a death registered on ONS between 1st January 2020 and 31st December 2020. #### Demographic Data Ethnicity: The primary care recorded ethnicity, supplemented where missing with ethnicity data from the Secondary Uses Service (SUS), was collapsed into the five high-level and 16 detailed census categories of White (White British, White Irish, other White), South Asian (Indian, Pakistani, Bangladeshi, other South Asian), Black (African, Caribbean, other Black), other (Chinese, all others), and mixed (White and Asian, White and African, White and Caribbean, other mixed) with an additional unknown ethnicity category included. Age: Patients’ age was calculated as of 30th June 2020 and grouped into 5 year bands. Sex: We used categories “male” and “female”, matching the ONS recorded categories; patients with any other/unknown sex were included as “unknown”. Deprivation: Deprivation was measured by the Index of Multiple Deprivation (IMD) derived from the patient’s postcode at lower super output area level. IMD was divided into quintiles, with higher values indicating greater deprivation. #### Causes of Death Patients were flagged if they had any death certified and registered in England or Wales between 1st January 2020 and 31st December 2020 and where applicable grouped into the 5 most common underlying causes of death (Table 1). View this table: [Table 1:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/T1) Table 1: Most common underlying causes of death occurring in England 2020 (26): ### Statistical methods We investigated the representativeness of TPP data by comparing OpenSAFELY-TPP-derived figures for 2020 with the following: (a) ONS IMD for all of England, (b) ONS age, sex (2020 estimates) and ethnicity (2011 census) across NHS England operating regions, and (c) causes of death (Malignant neoplasm of trachea, bronchus and lung, Ischaemic heart diseases, Dementia and Alzheimer disease, COVID-19 and Cerebrovascular diseases) in 2020 across NHS England operating regions. Proportions of each age group, sex, IMD band, ethnicity and cause of death were calculated and compared to the corresponding ONS data. For mortality analysis the denominator was the total number of deaths in 2020 and the numerator was the number of patients with the relevant ICD10 code (Table 1) as the underlying cause. TPP coverage was calculated as the proportion of TPP registered patients compared to ONS estimated populations within each Nomenclature of Territorial Units for Statistics (NUTS 1) region. ### Software and Reproducibility Data management was performed using Python 3.8, with analysis carried out using R. Code for data management and analysis as well as codelists are openly available online at ([https://github.com/opensafely/representativeness](https://github.com/opensafely/representativeness)) for inspection and re-use by anyone. ### Patient and Public Involvement We have developed a publicly available website [https://opensafely.org/](https://opensafely.org/) through which we invite any patient or member of the public to contact us regarding this study or the broader OpenSAFELY project. ## Results ### TPP coverage The population of active TPP patients (alive and registered on 30th June 2020) was 24 million representing 42.6% of the total UK population (based on the UK 2020 mid-year population estimate of 56 million). TPP coverage as a proportion of the ONS population was highest in the East of England (90.5%) and East Midlands (86.1%) and lowest in the West Midlands (16.8%), South East England (17.6%) and London (18.7%) (Figure 1). ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F1) Figure 1: Population coverage map showing coverage of each Nomenclature of Territorial Units for Statistics (NUTS-1) region. Population coverage based on ONS estimates covered by TPP with the number of patients in TPP per region. ### IMD Overall the proportion of IMD groups was similar, with only small differences between the TPP and ONS populations: In those with a recorded IMD there was a slightly higher proportion of TPP patients in the most deprived IMD group 1 (20.5%) and IMD group 3 (21.1) compared to national ONS estimates (20.0 and 20.3 respectively). TPP practices underrepresented patients in the least deprived IMD group 5 (18.3%) compared to ONS (19.4%) (Figure 2). IMD was missing for 2.3% of the TPP records. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F2) Figure 2: Barplot showing the proportion of ONS and TPP populations per IMD Quintile. The TPP population excludes 2.3% of patients without a recorded IMD. ### Sex For those with sex recorded as either male or female there was a similar proportion of women in the English population (50%) compared to ONS (50.1%) (Figure 3). The South West of England had the highest proportion of Females in TPP (50.8%) with London having the lowest proportion (48.8%). The difference in proportion of women between TPP and ONS estimates was within 0.1 percentage points for all regions (Figure 4). ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F3) Figure 3: Barplot showing the proportion of ONS and TPP populations by Sex ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F4.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F4) Figure 4: Barplot showing the proportion of ONS and TPP populations by Sex per NUTS-1 region ### Age There was a higher proportion of TPP patients in the age range 25-59 compared to ONS nationally, with a lower proportion of those under 25 years old (Figure 5). The difference in age distribution between TPP and ONS estimates was highest in London and the age distribution of the South West most closely resembled the ONS estimates (Figure 6). ![Figure 5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F5.medium.gif) [Figure 5:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F5) Figure 5: Cumulative frequency graph of ONS and TPP populations by age band ![Figure 6:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F6.medium.gif) [Figure 6:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F6) Figure 6: Cumulative frequency graph of ONS and TPP populations by age band per NUTS-1 region ### Age and sex Across England as a whole the higher proportion of TPP patients in the 35-59 age range compared to ONS estimates was largely due to a higher proportion of men in this age group in TPP. There was a higher proportion of women aged 20-29 in TPP and a lower proportion of men aged 20-29 compared to ONS estimates (Figures 7, 8). ![Figure 7:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F7.medium.gif) [Figure 7:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F7) Figure 7: Barplot showing the proportion of ONS and TPP populations by sex and age band ![Figure 8:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F8.medium.gif) [Figure 8:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F8) Figure 8: Barplot showing the proportion of ONS and TPP populations by sex and age band per NUTS-1 region ### Causes of death Across England there was a lower proportion of all five of the leading causes of deaths in TPP compared with ONS data (Figure 9). The biggest difference was in COVID-19 (12.2% in TPP, 12.9% in ONS) and the smallest difference was in Malignant neoplasm of trachea, bronchus and lung (4.9% in TPP, 5.0% in ONS). The difference in proportions of all 5 leading causes of death compared to ONS varied by region (Figure 10). COVID was overrepresented in TPP in all regions other than the North West (14.9% in TPP, 14.9% in ONS) and South East (7.5%, 10.0%). ![Figure 9:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F9.medium.gif) [Figure 9:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F9) Figure 9: Barplot showing the proportion of the 5 most common causes of deaths occuring in ONS and TPP in 2020. ![Figure 10:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F10.medium.gif) [Figure 10:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F10) Figure 10: Barplot showing the proportion of the 5 most common causes of deaths occuring in ONS and TPP in 2020 per NUTS-1 region ### Ethnicity (5 Groups) Of those with a recorded ethnicity the proportion of each ethnic group was within 1 percentage point of the ONS estimate across England as a whole for the 5 group ethnicity (Figure 11A, 12). The White population was underrepresented in all regions other than the North West (93.3%, 90.2%) (Figure 13). The Asian population was overrepresented in all regions other than the North West (3.5%, 5.5%) and South East (3.9%, 4.6%) (Figure 14). Ethnicity was not recorded for 9.4% of the TPP population. ![Figure 11:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F11.medium.gif) [Figure 11:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F11) Figure 11: Barplot showing the proportion of ONS and TPP populations per ethnicity grouped into A) 5 and B) 16 groups. The TPP population excludes the 9.4% without a recorded ethnicity. ![Figure 12:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F12.medium.gif) [Figure 12:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F12) Figure 12: Barplot showing the proportion of ONS and TPP populations per ethnicity (excluding White group) ![Figure 13:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F13.medium.gif) [Figure 13:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F13) Figure 13: Barplot showing the proportion of ONS and TPP populations per ethnicity grouped into 5 groups per NUTS-1 region ![Figure 14:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F14.medium.gif) [Figure 14:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F14) Figure 14: Barplot showing the proportion of ONS and TPP populations per ethnicity grouped into 5 groups per NUTS-1 region (excluding White group) ### Ethnicity (16 groups) Of those with a recorded ethnicity there was a lower proportion of White British people in TPP (74.8%) compared to ONS (79.8%) and higher proportion of Other White patients (9.6% TPP, 4.7% ONS). There was a lower proportion of both African (1.5%, 1.8%) and Caribbean (0.6%, 1.1%) patients and a higher proportion of Other black patients (0.6%,0.5%) (Figure 11B). There was clear regional variation in both the ethnic makeup of populations and the representativeness of ethnicity in TPP (Figure 15). ![Figure 15:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/06/23/2022.06.23.22276802/F15.medium.gif) [Figure 15:](http://medrxiv.org/content/early/2022/06/23/2022.06.23.22276802/F15) Figure 15: Barplot showing the proportion of ONS and TPP populations per ethnicity grouped into 16 groups per NUTS-1 region ## Discussion ### Summary This study has shown that TPP data made available via OpenSAFELY-TPP is broadly representative of the English population. Though there is high regional variability in the coverage of the OpenSAFEY-TPP data amongst English general practices, we nonetheless found broad similarity within regions, with only occasional discrepancies which should be considered when designing studies and interpreting outcomes from OpenSAFELY-TPP. Particularly notable was the over-representation of 25-50 year olds in London for both males and females. This may have contributed to a slight overall under-representation of under 25 year olds and over-representation of 25 -59 year olds nationally. We await the 2021 Census results as the assumption that ONS mid-year estimates nearly 10 years after the Census are more accurate than the TPP data may not be true. ### Strengths and weaknesses This study provides an overview of the representativeness of the OpenSAFELY-TPP cohort with regard to a variety of key characteristics in comparison with the general UK. The key strengths of this study are the use of high quality data from the EHR of all patients registered with a TPP practice which enabled us to compare participation rates for key sociodemographic characteristics (age, sex, IMD, and geographic location) and the comparison of ONS data held in OpenSAFELY-TPP to the matching national ONS data. The true population of England is not known (12), so we compared the OpenSAFELY-TPP data to ONS estimates as it forms the official population estimates of the UK (13). However, ONS estimates may over- or under-estimate population sizes based on regional differences. The mid-year population estimates also have limitations; they heavily rely on the use of health service data to estimate internal migration, in a process that may not be detecting all aspects of population change (12,13). The total GP registered population is always higher than the ONS population nationally, possibly due to over-counting in GP practice registers; under-counting in population estimates and different definitions of who counts as ‘resident’ in the country, but can be lower in certain regions (12). We have, therefore, probably overestimated the overall percentage of the total UK population covered by TPP (42.6%). Compared to the total GP registered population (60 million)(14), TPP covers 39.9% of the UK population. If we considered smaller geographic regions it would be possible to get areas with over 100% coverage. OpenSAFELY-TPP may have included deaths that were registered in Wales for patients registered in an English practice using TPP, whereas the ONS data was restricted to deaths registered in England. The most up-to-date formal estimates of the population by ethnic group currently available are from the 2011 Census. The ethnic makeup may have changed substantially from this point. OpenSAFELY-TPP was missing ethnicity for 10% of patients, and the missingness of ethnicity data in EHRs may not be random (8). The 2011 census used multiple imputation to account for missing ethnicity (15). We investigated the top 5 causes of death (accounting for ∼45%of all deaths) but did not look at others, or at health status more generally e.g. number of long term conditions. Regions are very large and more detailed regional analysis would be informative. We looked at one point in time and representativeness could change with time (e.g. TPP may take on or lose practices) or vary from year to year (e.g. the vaccine campaign may have prompted duplicated patients to be identified and deregistered). ### Findings in Context Over 50 studies have been conducted using the OpenSafely framework. However, the sheer scale of data made available in OpenSAFELY-TPP alone does not guarantee that the findings are generalisable to the English population at large. While at least one study has shown the large degree of geographical variation in coverage between EHR software providers (16), to our knowledge this is the first time that the representativeness of the population covered by the EHR software provider TPP has been systematically reported. We found that patients in TPP practices are broadly representative of England in terms of age, sex, IMD and ethnicity. The proportion of the five leading causes of death was broadly similar to those reported by ONS. The importance of a representative sample depends on the study question (17,18): careful consideration of this issue is warranted at the design, analysis and interpretation stage of every epidemiological study (19). We have previously described how differences in the design of EHR user interfaces can affect clinical coding (3) and the prescribing of certain medications (20,21). Additionally investigators may wish to consider representativeness of TPP when assessing variation in delivery of healthcare services due to variation in NHS service delivery and TPPs geographical coverage. ### Policy Implications and Interpretation The breadth of coverage provided by OpenSAFELY-TPP, 24 million patients across England representing 43% of the total English population (based on the ONS 2020 mid-year population estimate of 56 million), provides an unprecedented opportunity to support urgent research into the COVID-19 emergency. Users of OpenSAFELY must consider the issues of representativeness, generalisability and external validity associated with using TPP data for health research: overall, as this analysis shows, TPP registered patients are a representative sample of the English population as a whole. This paper is principally to inform interpretation of the numerous analyses completed and published using OpenSAFELY-TPP. However, OpenSAFELY is now also implemented in the data analysis environment of EMIS, a primary care electronic health record system supplier covering 55% of all practices in England. In addition, OpenSAFELY also supports federated analytics, where the same data preparation and analysis code is sent to OpenSAFELY-TPP and OpenSAFELY-EMIS to execute the same curation and analysis in each setting, with the results then combined into a single analysis, with a variety of papers already published using this approach (1–5). Nonetheless in the future it may still be more convenient or proportionate to execute analyses only in OpenSAFELY-TPP; therefore the high degree of representativeness demonstrated in this paper provides strong reassurance that such analyses present no interpretive or generalisability challenges. ## Conclusions Despite regional variations, data from OpenSAFELY-TPP is largely representative of the general population of England in terms of IMD, age, sex, ethnicity and causes of death. Following the use of OpenSAFELY-TPP data for a large number of COVID-19 studies since March 2020, it is reassuring to find that the overall representativeness of the population in the datasets is high. ## Data Availability Access to the underlying identifiable and potentially re-identifiable pseudonymised electronic health record data is tightly governed by various legislative and regulatory frameworks, and restricted by best practice. The data in OpenSAFELY is drawn from General Practice data across England where TPP is the Data Processor. TPP developers (CB, JC, JP, FH, and SH) initiate an automated process to create pseudonymised records in the core OpenSAFELY database, which are copies of key structured data tables in the identifiable records. These are linked onto key external data resources that have also been pseudonymised via SHA-512 one-way hashing of NHS numbers using a shared salt. DataLab developers and PIs (BG, LS, CEM, SB, AJW, KW, WJH, HJC, DE, PI, SD, GH, BBC, RMS, ID, KB, SE, EJW and CTR) holding contracts with NHS England have access to the OpenSAFELY pseudonymised data tables as needed to develop the OpenSAFELY tools. These tools in turn enable researchers with OpenSAFELY Data Access Agreements to write and execute code for data management and data analysis without direct access to the underlying raw pseudonymised patient data, and to review the outputs of this code. ## Administrative ## Conflicts of Interest All authors have completed the ICMJE uniform disclosure form at [www.icmje.org/coi\_disclosure.pdf](http://www.icmje.org/coi_disclosure.pdf) and declare the following: BG has received research funding from the Laura and John Arnold Foundation, the NHS National Institute for Health Research (NIHR), the NIHR School of Primary Care Research, the NIHR Oxford Biomedical Research Centre, the Mohn-Westlake Foundation, NIHR Applied Research Collaboration Oxford and Thames Valley, the Wellcome Trust, the Good Thinking Foundation, Health Data Research UK, the Health Foundation, the World Health Organisation, UKRI, Asthma UK, the British Lung Foundation, and the Longitudinal Health and Wellbeing strand of the National Core Studies programme; he also receives personal income from speaking and writing for lay audiences on the misuse of science. IJD has received unrestricted research grants and holds shares in GlaxoSmithKline (GSK). ## Funding This work was jointly funded by UKRI [COV0076;MR/V015737/1] NIHR and Asthma UK-BLF and the Longitudinal Health and Wellbeing strand of the National Core Studies programme.The OpenSAFELY data science platform is funded by the Wellcome Trust. BG’s work on better use of data in healthcare more broadly is currently funded in part by: the Wellcome Trust, NIHR Oxford Biomedical Research Centre, NIHR Applied Research Collaboration Oxford and Thames Valley, the Mohn-Westlake Foundation; all DataLab staff are supported by BG’s grants on this work. LS reports grants from Wellcome, MRC, NIHR, UKRI, British Council, GSK, British Heart Foundation, and Diabetes UK outside this work. AS is employed by LSHTM on a fellowship sponsored by GSK. KB holds a Wellcome Senior Research Fellowship (220283/Z/20/Z). BMK is also employed by NHS England working on medicines policy and clinical lead for primary care medicines data. The views expressed are those of the authors and not necessarily those of the NIHR, NHS England, Public Health England or the Department of Health and Social Care. Funders had no role in the study design, collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. ## Information governance and ethical approval NHS England is the data controller; TPP is the data processor; and the researchers on OpenSAFELY are acting with the approval of NHS England. This implementation of OpenSAFELY is hosted within the TPP environment which is accredited to the ISO 27001 information security standard and is NHS IG Toolkit compliant;(22,23) patient data has been pseudonymised for analysis and linkage using industry standard cryptographic hashing techniques; all pseudonymised datasets transmitted for linkage onto OpenSAFELY are encrypted; access to the platform is via a virtual private network (VPN) connection, restricted to a small group of researchers; the researchers hold contracts with NHS England and only access the platform to initiate database queries and statistical models; all database activity is logged; only aggregate statistical outputs leave the platform environment following best practice for anonymisation of results such as statistical disclosure control for low cell counts.(24) The OpenSAFELY research platform adheres to the obligations of the UK General Data Protection Regulation (GDPR) and the Data Protection Act 2018. In March 2020, the Secretary of State for Health and Social Care used powers under the UK Health Service (Control of Patient Information) Regulations 2002 (COPI) to require organisations to process confidential patient information for the purposes of protecting public health, providing healthcare services to the public and monitoring and managing the COVID-19 outbreak and incidents of exposure; this sets aside the requirement for patient consent.(25) Taken together, these provide the legal bases to link patient datasets on the OpenSAFELY platform. GP practices, from which the primary care data are obtained, are required to share relevant health information to support the public health response to the pandemic, and have been informed of the OpenSAFELY analytics platform. This study was approved by the Health Research Authority (REC reference 20/LO/0651) and by the LSHTM Ethics Board (reference 21863). ## Data access and verification Access to the underlying identifiable and potentially re-identifiable pseudonymised electronic health record data is tightly governed by various legislative and regulatory frameworks, and restricted by best practice. The data in OpenSAFELY is drawn from General Practice data across England where TPP is the Data Processor. TPP developers (CB, JC, JP, FH, and SH) initiate an automated process to create pseudonymised records in the core OpenSAFELY database, which are copies of key structured data tables in the identifiable records. These are linked onto key external data resources that have also been pseudonymised via SHA-512 one-way hashing of NHS numbers using a shared salt. DataLab developers and PIs (BG, LS, CEM, SB, AJW, KW, WJH, HJC, DE, PI, SD, GH, BBC, RMS, ID, KB, SE, EJW and CTR) holding contracts with NHS England have access to the OpenSAFELY pseudonymised data tables as needed to develop the OpenSAFELY tools. These tools in turn enable researchers with OpenSAFELY Data Access Agreements to write and execute code for data management and data analysis without direct access to the underlying raw pseudonymised patient data, and to review the outputs of this code. All code for the full data management pipeline—from raw data to completed results for this analysis—and for the OpenSAFELY platform as a whole is available for review at [github.com/OpenSAFELY](http://github.com/OpenSAFELY). The data management and analysis code for this paper was led by CDA and contributed to by WJH and AJW. ## Guarantor BG is guarantor. ## Contributorship *All authors contributed to and approved the final manuscript*. ## Acknowledgements We are very grateful for all the support received from the TPP Technical Operations team throughout this work, and for generous assistance from the information governance and database teams at NHS England / NHSX. ## Abbreviations EHR : Electronic Health Record ICD : International Classification of Diseases IMD : Index of Multiple Deprivation NUTS : Nomenclature of Territorial Units for Statistics ONS : Office for National Statistics SUS : Secondary Uses Service * Received June 23, 2022. * Revision received June 23, 2022. * Accepted June 23, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. 1.Curtis HJ, Inglesby P, MacKenna B, Croker R, Hulme W, Rentsch CT, et al. Recording of “COVID-19 vaccine declined” among vaccination priority groups: a cohort study on 57.9 million NHS patients’ primary care records in situ using OpenSAFELY [Internet]. bioRxiv. medRxiv; 2021. Available from: [https://www.medrxiv.org/content/10.1101/2021.08.05.21259863v1.abstract](https://www.medrxiv.org/content/10.1101/2021.08.05.21259863v1.abstract) 2. 2.The OpenSAFELY Collaborative, Curtis HJ, Inglesby P, Morton CE, MacKenna B, Walker AJ, et al. Trends and clinical characteristics of COVID-19 vaccine recipients: a federated analysis of 57.9 million patients’ primary care records in situ using OpenSAFELY [Internet]. bioRxiv. medRxiv; 2021. Available from: [http://medrxiv.org/lookup/doi/10.1101/2021.01.25.21250356](http://medrxiv.org/lookup/doi/10.1101/2021.01.25.21250356) 3. 3.Walker AJ, MacKenna B, Inglesby P, Tomlinson L, Rentsch CT, Curtis HJ, et al. Clinical coding of long COVID in English primary care: a federated analysis of 58 million patient records in situ using OpenSAFELY. Br J Gen Pract. 2021 Nov;71(712):e806–14. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNzEvNzEyL2U4MDYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wNi8yMy8yMDIyLjA2LjIzLjIyMjc2ODAyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 4. 4.OpenSAFELY Collaborative, Fisher L, Speed V, Curtis HJ, Rentsch CT, Wong AYS, et al. Potentially inappropriate prescribing of DOACs to people with mechanical heart valves: A federated analysis of 57.9 million patients’ primary care records in situ using OpenSAFELY. Thromb Res. 2022 Mar;211:150–3. 5. 5.Fisher L, Hopcroft LEM, Rodgers S, Barrett J, Oliver K, Avery AJ, et al. Changes in English medication safety indicators throughout the COVID-19 pandemic: a federated analysis of 57 million patients’ primary care records in situ using OpenSAFELY [Internet]. bioRxiv. 2022. Available from: [https://www.medrxiv.org/content/10.1101/2022.05.05.22273234v1.abstract](https://www.medrxiv.org/content/10.1101/2022.05.05.22273234v1.abstract) 6. 6.Herrett E, Gallagher AM, Bhaskaran K, Forbes H, Mathur R, van Staa T, et al. Data Resource Profile: Clinical Practice Research Datalink (CPRD). Int J Epidemiol. 2015 Jun;44(3):827–36. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyv098&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26050254&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F06%2F23%2F2022.06.23.22276802.atom) 7. 7. The OpenSAFELY Collaborative, Williamson E, Walker AJ, Bhaskaran K, Bacon S, Bates C, et al. OpenSAFELY: factors associated with COVID-19-related hospital death in the linked electronic health records of 17 million adult NHS patients [Internet]. bioRxiv. medRxiv; 2020. Available from: [http://medrxiv.org/lookup/doi/10.1101/2020.05.06.20092999](http://medrxiv.org/lookup/doi/10.1101/2020.05.06.20092999) 8. 8.Mathur R, Bhaskaran K, Chaturvedi N, Leon DA, vanStaa T, Grundy E, et al. Completeness and usability of ethnicity data in UK-based primary care and hospital databases. J Public Health. 2014 Dec;36(4):684–92. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/pubmed/fdt116&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24323951&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F06%2F23%2F2022.06.23.22276802.atom) 9. 9.Steel N, Ford JA, Newton JN, Davis ACJ, Vos T, Naghavi M, et al. Changes in health in the countries of the UK and 150 English Local Authority areas 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2018 Nov 3;392(10158):1647–61. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F06%2F23%2F2022.06.23.22276802.atom) 10. 10.Mid-year population estimates QMI [Internet]. [cited 2022 Jan 14]. Available from: [https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/methodologies/midyearpopulationestimatesqmi](https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/methodologies/midyearpopulationestimatesqmi) 11. 11.Nomis - Nomis - Official Labour Market Statistics [Internet]. [cited 2021 Dec 13]. Available from: [https://www.nomisweb.co.uk/](https://www.nomisweb.co.uk/) 12. 12.Baker C. Population estimates & GP registers: why the difference? 2016 Dec 12 [cited 2021 Dec 16]; Available from: [https://commonslibrary.parliament.uk/population-estimates-gp-registers-why-the-difference/](https://commonslibrary.parliament.uk/population-estimates-gp-registers-why-the-difference/) 13. 13.Park N. Population estimates for the UK, England and Wales, Scotland and Northern Ireland - Office for National Statistics [Internet]. Office for National Statistics; 2021 [cited 2021 Dec 16]. Available from: [https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/bulletins/annualmidyearpopulationestimates/mid2020](https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/bulletins/annualmidyearpopulationestimates/mid2020) 14. 14.Patients Registered at a GP Practice January 2020; Special Topic [Internet]. NHS Digital. [cited 2022 Jan 28]. Available from: [https://digital.nhs.uk/data-and-information/publications/statistical/patients-registered-at-a-gp-practice/january-2020](https://digital.nhs.uk/data-and-information/publications/statistical/patients-registered-at-a-gp-practice/january-2020) 15. 15.Wardman L, Aldrich S, Rogers S. Census item edit and imputation process. Disponible en ligne:[06/01/2015] [http://www.onsgovuk/ons/guide-method/census/2011/census-data/2011-census-userguide/quality-and-methods/quality/quality-measures/response-and-imputation-rates/item-edit-andimputation-processpdf](http://www.onsgovuk/ons/guide-method/census/2011/census-data/2011-census-userguide/quality-and-methods/quality/quality-measures/response-and-imputation-rates/item-edit-andimputation-processpdf). 2011; 16. 16.Kontopantelis E, Stevens RJ, Helms PJ, Edwards D, Doran T, Ashcroft DM. Spatial distribution of clinical computer systems in primary care in England in 2016 and implications for primary care electronic medical record databases: a cross-sectional population study. BMJ Open. 2018 Feb 28;8(2):e020738. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC8yL2UwMjA3MzgiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wNi8yMy8yMDIyLjA2LjIzLjIyMjc2ODAyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 17. 17.Rothman KJ, Gallacher JEJ, Hatch EE. Why representativeness should be avoided. Int J Epidemiol. 2013 Aug;42(4):1012–4. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dys223&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24062287&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F06%2F23%2F2022.06.23.22276802.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000325167800013&link_type=ISI) 18. 18.Richiardi L, Pizzi C, Pearce N. Commentary: Representativeness is usually not necessary and often should be avoided. Int J Epidemiol. 2013 Aug;42(4):1018–22. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyt103&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24062290&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F06%2F23%2F2022.06.23.22276802.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000325167800016&link_type=ISI) 19. 19.Ebrahim S, Davey Smith G. Commentary: Should we always deliberately be non-representative? Int J Epidemiol. 2013 Aug;42(4):1022–6. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyt105&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24062291&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F06%2F23%2F2022.06.23.22276802.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000325167800017&link_type=ISI) 20. 20.MacKenna B, Bacon S, Walker AJ, Curtis HJ, Croker R, Goldacre B. Impact of Electronic Health Record Interface Design on Unsafe Prescribing of Ciclosporin, Tacrolimus, and Diltiazem: Cohort Study in English National Health Service Primary Care. J Med Internet Res. 2020 Oct 16;22(10):e17003. 21. 21.MacKenna B, Curtis HJ, Walker AJ, Bacon S, Croker R, Goldacre B. Suboptimal prescribing behaviour associated with clinical software design features: a retrospective cohort study in English NHS primary care. Br J Gen Pract. 2020 Sep;70(698):e636–43. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNzAvNjk4L2U2MzYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wNi8yMy8yMDIyLjA2LjIzLjIyMjc2ODAyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 22. 22.BETA – Data Security Standards - NHS Digital [Internet]. NHS Digital. [cited 2020 Apr 30]. Available from: [https://digital.nhs.uk/about-nhs-digital/our-work/nhs-digital-data-and-technology-standards/framework/beta\---|data-security-standards](https://digital.nhs.uk/about-nhs-digital/our-work/nhs-digital-data-and-technology-standards/framework/beta\---|data-security-standards) 23. 23.Data Security and Protection Toolkit - NHS Digital [Internet]. NHS Digital. [cited 2020 Apr 30]. Available from: [https://digital.nhs.uk/data-and-information/looking-after-information/data-security-and-information-governance/data-security-and-protection-toolkit](https://digital.nhs.uk/data-and-information/looking-after-information/data-security-and-information-governance/data-security-and-protection-toolkit) 24. 24.ISB1523: Anonymisation Standard for Publishing Health and Social Care Data - NHS Digital [Internet]. NHS Digital. [cited 2020 Apr 30]. Available from: [https://digital.nhs.uk/data-and-information/information-standards/information-standards-and-data-collections-including-extractions/publications-and-notifications/standards-and-collections/isb1523-anonymisation-standard-for-publishing-health-and-social-care-data](https://digital.nhs.uk/data-and-information/information-standards/information-standards-and-data-collections-including-extractions/publications-and-notifications/standards-and-collections/isb1523-anonymisation-standard-for-publishing-health-and-social-care-data) 25. 25.Secretary of State for Health and Social Care - UK Government. Coronavirus (COVID-19): notification to organisations to share information [Internet]. 2020. Available from: [https://web.archive.org/web/20200421171727/](https://web.archive.org/web/20200421171727/) [https://www.gov.uk/government/publications/coronavirus-covid-19-notification-of-data-controllers-to-share-information](https://www.gov.uk/government/publications/coronavirus-covid-19-notification-of-data-controllers-to-share-information) 26. 26.Mortality statistics - underlying cause, sex and age - Nomis - Official Labour Market Statistics [Internet]. [cited 2022 Jan 28]. Available from: [https://www.nomisweb.co.uk/datasets/mortsa](https://www.nomisweb.co.uk/datasets/mortsa)