The COVID-19 Yorkshire Rehabilitation Scale (C19-YRS): application and psychometric analysis in a post-COVID-19 syndrome cohort
===============================================================================================================================

* Nick Preston
* Amy Parkin
* Sophie Makower
* Denise H Ross
* Jeremy Gee
* Stephen J Halpin
* Mike C Horton
* Rory J O’Connor
* Manoj Sivan

## Abstract

**Background** As our understanding of the nature and prevalence of Post-COVID-19 syndrome (PCS) is increasing, a measure of the impact of COVID-19 could provide valuable insights into patients’ perceptions in clinical trials and epidemiological studies, as well as routine clinical practice.

**Objective** To evaluate the clinical usefulness and psychometric properties of the COVID-19 Yorkshire Rehabilitation Scale (C19-YRS) in patients with PCS.

**Design** A prospective, observational study of 187 consecutive patients attending a post-COVID-19 rehabilitation clinic. The C19-YRS was used to record patients’ symptoms, functioning and disability. A global health question was used to measure the overall impact of PCS on health. Classical psychometric methods (data quality, scaling assumptions, targeting, reliability and validity) were used to assess the C19-YRS.

**Results** For the overall scale, missing data were low, scaling and targeting assumptions were satisfied, and internal consistency was high (Cronbach’s alpha = 0.891). Relationships between perception of health and patients’ reports of symptoms, functioning and disability demonstrated good concordance.

**Conclusions** This is the first study to examine the psychometric properties of an outcome measure in patients with PCS. In this sample of patients, the C19-YRS was clinically useful and satisfied standard psychometric criteria, providing preliminary evidence of its suitability as a measure of PCS.

Key words
*   SARS CoV-2
*   Post-COVID-19 symptoms
*   Psychometrics
*   Long Covid
*   Patient Reported Outcome Measure (PROM)

## Introduction

The medium and long-term problems experienced by survivors of COVID-19 are emerging, but standardized assessments of functioning, disability and health are lacking. Long-term symptoms of COVID-19 might be predicted from the previous coronavirus outbreaks in 2002 and 2012 – Severe Acute Respiratory Syndrome (SARS) and Middle East Respiratory Syndrome (MERS) respectively. A meta-analysis of follow-up studies demonstrated that 25% of hospitalized survivors of SARS and MERS experienced reduced lung function and lower exercise capacity six months post-discharge.1 One year on, post-traumatic stress disorder (PTSD), depression, anxiety and reduced quality of life were observed. Preliminary research suggests that the impact of COVID-19 infection is similar.2

Over one million people in the UK who contracted COVID-19 report health problems more than four weeks after the onset of the acute illness. In addition, almost 700,000 people report ongoing symptoms more than 12 weeks after contracting COVID-19, a condition now referred to as ‘Long Covid’ (LC) or Post-COVID-19 Syndrome (PCS).3

The most common symptoms of PCS include fatigue, dyspnea, pain, anxiety and cognitive problems, but there are over 200 reported symptoms affecting ten organ systems.4 One study following 143 individuals seven weeks post-discharge found 53% of patients reported fatigue, 43% dyspnea and 27% joint pain.4 A significant number of patients report limitations with their activities of daily living (ADL), with almost 130,000 patients stating that these limitations are severe.3

Until recently there was no psychometrically robust patient-reported measure that focused on the impact of PCS. The COVID-19 Yorkshire Rehabilitation Scale (C19-YRS) is a 22-item patient-reported outcome measure designed to evaluate the long-term impact of COVID-19. The C19-YRS now includes clinician-completed, self-report and digital versions.5 Content validity of the C19-YRS has been demonstrated,2 and the C19-YRS is now used in the UK’s first specialist PCS community rehabilitation service,6 and 26 other UK National Health Service (NHS) PCS rehabilitation services. This paper describes the first stage in establishing the psychometric properties of the C19-YRS as an outcome measure for PCS.

## Methods

### Participants and Recruitment

This was a prospective observational study on patients attending a PCS Community Rehabilitation Service covering the Leeds City Region, a mixed urban and rural district in the North of England with a population of approximately 850,000 people, which includes areas of significant social deprivation. Patients were referred by their General Practitioner (GP), Community Matron or Respiratory Physiotherapy team to a PCS Community Rehabilitation Service and completed a self-report C19-YRS as part of initial triage. Data are collected in the service as part of routine clinical evaluation and ethical approval for secondary analysis of anonymized data by the University of Leeds research team was obtained (MREC 20-041).

### The C19-YRS

The C19-YRS consists of 22 items with each item scored on an 11-point numerical rating scale from 0 (none of this symptom) to 10 (extremely severe level or impact). The C19-YRS is divided into four sub-scales (range of total score for each sub-scale): symptom severity score (0 to 100), functional disability score (0 to 50), additional symptoms (0 to 60) and overall health (0 to 10). The C19-YRS was completed independently by each patient or, if the patient preferred, by a researcher or member of the administrative team via telephone. Patients’ family members or carers were permitted to help complete the responses.

On completion of clinical care for each patient, anonymized data from each completed C19-YRS was transferred to an Excel spreadsheet by clinician researchers who were not involved in the initial assessment pathway and follow-up of the patients.

### Data Analysis

Descriptive statistics are presented as arithmetic mean and SD, or median and inter-quartile range as appropriate. Absolute and relative frequencies as appropriate for demographic and categorical variables on the C19-YRS are presented. Analyses were carried out using IBM SPSS (Statistics 26, Release 26.0.0.0, 64-bit edition, IBM Corp.). Five psychometric analyses (data quality, scaling assumptions, targeting and reliability) were undertaken.

### Data Quality

Data quality concerns the extent to which a scale can be administered successfully in the target sample. The C19-YRS data were examined for percentage missing items and the percentage of the sample for whom total scores could be calculated.7 Imputed scores were not used for missing items.

### Scaling Assumptions

Tests of scaling assumptions examine whether it is legitimate to sum item scores to generate scale scores. In order for a set of items to be legitimately summed to form a total score, a series of criteria should be satisfied.8-10 We tested the C19-YRS against these criteria, which are:

1.  Items should be roughly parallel, that is, measure at the same point on the scale and have similar variance, otherwise they do not contribute equally to the variance of the total score.11 A set of items is considered parallel when their item response option frequency distributions, and their item mean scores and standard deviations are roughly similar.9

2.  Items should measure the same underlying construct, otherwise it is not appropriate to combine them to generate a total score.12 A set of items is considered to be measuring the same construct when each item’s corrected item-total correlation, which is the correlation between each item and the total score computed from the remaining items in that scale, exceeds 0.30.11

3.  Items in the scale should contain a similar proportion of information concerning the construct being measured. This criterion is considered satisfied the corrected item-total correlations exceed 0.30.13

### Targeting

Targeting refers to the match between the distribution of health problems in the sample and the range of health problems measured by the scale. The better this match, the greater the potential for precise measurement. Targeting was evaluated by examining floor and ceiling effects, score distributions, and skewness statistics. Floor effects are the percentage of patients scoring zero (symptom not present) and ceiling effects are the percentage of patients scoring 10 (most severe impact of symptom) on each item. It is recommended that floor and ceiling effects should be less than 20% each on each item.14

### Reliability

Reliability describes the extent to which scale scores are free from random error. Scales should generate reliable estimates of the construct being measured (internal consistency). Cronbach’s alpha coefficient was used to determine this criterion.15 Although a range of minimum values has been suggested, it is widely accepted that Cronbach’s alpha should exceed 0.80 for group comparison studies.10

### Validity

Validity was evaluated by examining the extent to which Pearson’s correlations between the sub-scales of the C19-YRS were consistent with expectations.16 It was predicted that correlations between the symptom severity, functional disability and additional symptoms sub-scales would be moderate (between 0.3 and 0.7) and exceed the correlation with the overall health score.

## Results

### Sample Characteristics

Data for the analyses were obtained from 188 consecutive assessments of PCS patients. Patient details are given in Table 1. One patient was removed from the analyses because a significant number of answers were missing, presumed to be an oversight of the respondent.

View this table:
[Table 1](http://medrxiv.org/content/early/2021/06/30/2021.06.28.21259613/T1)

Table 1 Patient demographics

Patients’ scores on the C19-YRS sub-scales are presented in Table 2. Most patients reported problems with their mental or physical health. Fatigue was the most common complaint, with 97.3% of patients reporting fatigue of varying severity, followed by the onset of pain not present before COVID-19 was contracted (94.3%). The most common new pain was muscle pain which affected 70% of patients, followed by headache (67%), chest (64%) and joint pain (59%). Approximately one third of patients also experienced new pain in their abdomen or other regions. Mental health problems were reported by 41% of patients, with 17% of these patients reporting respiratory or cardiac comorbidity. Respiratory or cardiac health issues, or both, were reported by 37% of patients. Swallowing, incontinence, skin rash and fever were bothersome for very few respondents.

View this table:
[Table 2](http://medrxiv.org/content/early/2021/06/30/2021.06.28.21259613/T2)

Table 2 Patients’ scores on the C19-YRS sub-scales

### Data Quality

Missing data for items were low (range 0.5 – 19.8%). Sub-scale scores could be calculated for 67% of patients reporting symptom severity, 82% of patients reporting functional disability, 83% of patients reporting additional symptoms and 98% of patients reporting overall health. Details of scores are given in Table 2.

### Scaling Assumptions

Item response option frequency distributions were symmetric. Item means and standard deviations were similar indicating that they were roughly parallel (Table 3), although there was a greater range in symptom severity. Corrected item-total correlations exceeded 0.30 for all items except swallowing (0.24), incontinence (0.28) and skin rash (0.14) indicating that scaling assumptions were met for most items, including fever (0.33).

View this table:
[Table 3](http://medrxiv.org/content/early/2021/06/30/2021.06.28.21259613/T3)

Table 3 Psychometric properties of the C19-YRS sub-scales

### Targeting

Scores spanned the range of the scale on admission and discharge and demonstrated good variability (Table 3). Results for some items demonstrated notable floor effects, especially for swallowing (72.7%), skin rash (66.8%), and fever (64.7%). There were no ceiling effects in any subscale.

### Reliability

Internal consistency of the overall C19-YRS was good (Cronbach’s alpha = 0.891). Individual sub-scales also demonstrated good reliability. Deletion of the items noted to have poor scaling assumptions and targeting improved the reliability of the symptom severity sub-scale (swallowing, incontinence removed; Cronbach’s alpha 0.79 to 0.81) and the additional symptoms sub-scale (fever, skin rash removed; Cronbach’s alpha 0.70 to 0.74).

### Validity

The symptom severity, functional disability and additional symptoms sub-scales correlated strongly with each other (Table 4), indicating that the sub-scales have a coherent internal structure. The overall health scale also correlated strongly with the other three subscales. As this is a more generic question of health status, and is an item commonly used in health-related quality of life measures, this provides some preliminary evidence of construct validity.

View this table:
[Table 4](http://medrxiv.org/content/early/2021/06/30/2021.06.28.21259613/T4)

Table 4 Correlation of the C19-YRS sub-scales with the overall health scale*

## Discussion

The C19-YRS was developed as a disease-specific patient-based measure of the impact of COVID-19 infection.17,18 The scale has been used successfully to gather symptom severity and functional impact and monitor progress in PCS, and is recommended by National Health Service England (NHSE)19 and the National Institute for Health and Care Excellence (NICE).20 However, it is recognized that the C19-YRS requires further iterations for development and refinement. In this first round of psychometric testing, the measurement properties of the C19-YRS were found to be good.

Many studies of rehabilitation in PCS have used generic measures of health outcome. Conceptually however, there are good arguments for making a PCS-specific scale given that many rehabilitation strategies aim to ameliorate the specific impairments associated with PCS. We examined this self-report version of the C19-YRS, initially designed for use with patients discharged from acute hospital settings, then modified to suit both hospitalized and non-hospitalized patients, to determine the stability of its psychometric properties and its potential as a measure of PCS. Our results provide evidence for that potential. In the group studied, evidence was found for data quality, scaling assumptions, targeting and reliability for most items. There is now evidence of psychometric stability across a wide sample of people with PCS. These findings from this study provide useful information and illustrate the strong potential of the C19-YRS to achieve the necessary standards for highly accurate, psychometrically robust measurement.

This study has limitations. First, it is a study from a single clinical site and includes patients with a diverse range of experiences. Whilst there is some evidence that small samples provide useful reliability and validity estimates,21 we recognize that our sample is relatively small at present. Nevertheless, our patient cohort is growing rapidly, and we aim to have in excess of 500 patients in our definitive psychometric analyses. Second, the scale is self-report and thus the extent to which it is applicable in patients with severe fatigue or who have impairments affecting communication remains to be determined. In this study, patients could be provided with assistance to complete the questionnaire, but is recognized that patients may answer items in questionnaires differently when the measures are self-completed compared to an interview by a member of staff, and this may lead to a bias in the reporting of the scores.22 Third, we have not studied test retest-reliability. However, Cronbach’s alpha is considered to be a conservative reliability estimate, and test-retest reliability often over-estimates reliability. The underpinning research for this has been discussed by Nunnally23 and others.10,22,23 Despite these limitations, we are confident that the C19-YRS will turn out to be a useful addition to current assessments of PCS in clinical studies, and could be used to complement clinician-scored measures. Furthermore, the items in the scale provide qualitative information to clinicians to assist in targeting their clinical interventions to individuals’ needs. It has advantages over other approaches, as it may be used in any setting, does not require an external rater, and is not laboratory based or require special equipment. Most importantly it measures patients’ perspectives.24

### Further Research

Subsequent psychometric testing will use Rasch analysis to determine whether the scale meets the fundamental axioms that define scientific measurement and permit the transformation of raw (ordinal) scores to interval level measurement.12 Further evaluations will examine the short- and long-term responsiveness of the scale to changes in symptom severity and the overall impact of rehabilitation on PCS. This will also determine the minimal clinically important difference (MCID) of the scale that correlates to clinical improvement or deterioration of the condition reported by patients.

## Conclusion

This is the first study to examine the psychometric properties of a PCS-specific outcome measure that captures and evaluates the symptoms experienced by patients. In this sample of patients, the C19-YRS was clinically useful and satisfied standard psychometric criteria. The C19-YRS shows good internal consistency, and scaling and targeting assumptions were satisfied. This provides preliminary evidence that the C19-YRS outcome measure of PCS has satisfactory psychometric properties.

## Data Availability

Unavailable

[https://www.unavailable](https://www.unavailable) 

## Acknowledgements

NHS England clinical guidance suggests use of the C19-YRS at first assessment, at six weeks and at six months in PCS.19 The National Institute for Health and Care Excellence (NICE) recommend use of the C19-YRS for comprehensive assessment of patients.20

This work is supported by a University of Leeds Medical Research Council (MRC) Confidence in Concept (CiC) grant for the psychometric evaluation of the C19-YRS. RJOC’s research is supported by the National Institute for Health Research (NIHR) infrastructure at Leeds and Sheffield. The views expressed are those of the authors and not necessarily those of the Medical Research Council, the NHS, NICE, the NIHR or the UK’s Department of Health and Social Care.

The C19-YRS can be freely obtained under license from the University of Leeds ([https://licensing.leeds.ac.uk/product/c19-yrs-covid-19-yorkshire-rehabilitation-scale](https://licensing.leeds.ac.uk/product/c19-yrs-covid-19-yorkshire-rehabilitation-scale)).

*   Received June 28, 2021.
*   Revision received June 28, 2021.
*   Accepted June 30, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.Ahmed H, Patel K, Greenwood DC, et al. Long-term clinical outcomes in survivors of severe acute respiratory syndrome and Middle East respiratory syndrome coronavirus outbreaks after hospitalisation or ICU admission: A systematic review and meta-analysis. Journal of rehabilitation medicine : official journal of the UEMS European Board of Physical and Rehabilitation Medicine 2020;52(5):jrm00063. doi: 10.2340/16501977-2694 [published Online First: 2020/05/26]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2340/16501977-2694&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 

2.  2.Halpin SJ, McIvor C, Whyatt G, et al. Postdischarge symptoms and rehabilitation needs in survivors of COVID-19 infection: A cross-sectional evaluation. Journal of Medical Virology 2021;93(2):1013-22. doi: [https://doi.org/10.1002/jmv.26368](https://doi.org/10.1002/jmv.26368)
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 

3.  3.Office for National Statistics. Prevalence of ongoing symptoms following coronavirus (COVID-19) infection in the UK: 1 April 2021 2021 [Available from: [https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/prevalenceofongoingsymptomsfollowingcoronaviruscovid19infectionintheuk/1april2021](https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/prevalenceofongoingsymptomsfollowingcoronaviruscovid19infectionintheuk/1april2021) accessed 18 May 2021.
    
    

4.  4.Davis HE, Assaf GS, McCorkell L, et al. Characterizing Long COVID in an International Cohort: 7 Months of Symptoms and Their Impact. medRxiv 2021:2020.12.24.20248802. doi: 10.1101/2020.12.24.20248802
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4xMi4yNC4yMDI0ODgwMnYzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDYvMzAvMjAyMS4wNi4yOC4yMTI1OTYxMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

5.  5.Sivan M S. H J. G, et al. The self-report version and digital format of the COVID-19 Yorkshire Rehabilitation Scale (C19-YRS) for Long Covid or Post-COVID syndrome assessment and monitoring. Advances in CLinical Neuroscience and Rehabilitation 2021;20(3):2–5. [published Online First: 13 April 2021]
    
    

6.  6.Leeds Clinical Commissioning Group. Leeds Covid-19 Recovery Follow-up and Guidance in Primary Care, 2020.
    
    

7.  7.Ware J, Snoww K, Ma K, et al. SF36 Health Survey: Manual and Interpretation Guide. Lincoln, RI: Quality Metric, Inc, 1993 1993;30
    
    

8.  8.Aaronson N, Alonso J, Burnam A, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11(3):193–205. doi: 10.1023/a:1015291021312 [published Online First: 2002/06/21]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1023/A:1015291021312&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12074258&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000175209700001&link_type=ISI) 

9.  9.Likert R. A technique for the measurement of attitudes. Archives of Psychology 1932;22 140:55–55.
    
    

10. 10.Nunnally JC, Bernstein IH. Psychometric Theory. 3 ed. New York: McGraw-Hill 1994.
    
    

11. 11.McHorney CA, Ware JE, Jr.., Lu JF, et al. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994;32(1):40–66. doi: 10.1097/00005650-199401000-00004 [published Online First: 1994/01/01]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00005650-199401000-00004&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=8277801&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1994MR08100004&link_type=ISI) 

12. 12.Tennant A, Conaghan PG. The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care & Research 2007;57(8):1358–62. doi: 10.1002/art.23108
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/art.23108&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18050173&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000251638200004&link_type=ISI) 

13. 13.MAP-R for windows: multitrait / multi-item analysis program - revised user’s guide; 1997.
    
    

14. 14.Holmes WC, Shea JA. Performance of a new, HIV/AIDS-targeted quality of life (HAT-QoL) instrument in asymptomatic seropositive individuals. Qual Life Res 1997;6(6):561–71. doi: 10.1023/a:1018464200708 [published Online First: 1997/08/01]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1023/A:1018464200708&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9330555&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997XX98000010&link_type=ISI) 

15. 15.Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16(3):297–334. doi: 10.1007/BF02310555
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/BF02310555&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1951YB33000004&link_type=ISI) 

16. 16.Campbell DT. Recommendations for APA test standards regarding construct, trait, or discriminant validity. Am Psychol 1960;15(8):546–53. doi: 10.1037/h0048255
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1037/h0048255&link_type=DOI) 

17. 17.Hobart JC, Riazi A, Lamping DL, et al. Measuring the impact of MS on walking ability: the 12-Item MS Walking Scale (MSWS-12). Neurology 2003;60(1):31–6. doi: 10.1212/wnl.60.1.31 [published Online First: 2003/01/15]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1212/WNL.60.1.31&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12525714&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 

18. 18.McGuigan C, Hutchinson M. Confirming the validity and responsiveness of the Multiple Sclerosis Walking Scale-12 (MSWS-12). Neurology 2004;62(11):2103–5. doi: 10.1212/01.wnl.0000127604.84575.0d [published Online First: 2004/06/09]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1212/01.WNL.0000127604.84575.0D&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15184625&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 

19. 19.NHS England. National guidance for post-COVID syndrome assessment clinics. London: NHS England and NHS Improvement, 2021.
    
    

20. 20.National Institute for Health and Care Excellence. COVID-19 rapid guideline: managing the long-term effects of COVID-19 2021 [Available from: [https://www.nice.org.uk/guidance/ng188/chapter/common-symptoms-of-ongoing-symptomatic-covid-19-and-post-covid-19-syndrome#common-symptoms-of-ongoing-symptomatic-covid-19-and-post-covid-19-syndrome](https://www.nice.org.uk/guidance/ng188/chapter/common-symptoms-of-ongoing-symptomatic-covid-19-and-post-covid-19-syndrome#common-symptoms-of-ongoing-symptomatic-covid-19-and-post-covid-19-syndrome) accessed 18 May 2021.
    
    

21. 21.Hobart JC, Cano SJ, Warner TT, et al. What sample sizes for reliability and validity studies in neurology? J Neurol 2012;259(12):2681–94. doi: 10.1007/s00415-012-6570-y [published Online First: 2012/06/26]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00415-012-6570-y&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22729386&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000311674300022&link_type=ISI) 

22. 22.Korner-Bitensky N, Wood-Dauphinee S, Siemiatycki J, et al. Health-related information postdischarge: telephone versus face-to-face interviewing. Archives of physical medicine and rehabilitation 1994;75(12):1287–96. [published Online First: 1994/12/01]
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=7993166&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1994PX23300003&link_type=ISI) 

23. 23.Nunnally JC. Introduction to psychological measurement. New York: McGraw Hill 1970.
    
    

24. 24.Bergner M, Rothman ML. Health status measures: an overview and guide for selection. Annu Rev Public Health 1987;8:191–210. doi: 10.1146/annurev.pu.08.050187.001203 [published Online First: 1987/01/01]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1146/annurev.pu.08.050187.001203&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=3555521&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F06%2F30%2F2021.06.28.21259613.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1987H268100008&link_type=ISI)