Development and validation of the long covid symptom and impact tools, a set of patient-reported instruments constructed from patients’ lived experience

Viet-Thi Tran, Caroline Riveros, Bérangère Clepier, Moïse Desvarieux, Camille Collet, Youri Yordanov, Philippe Ravaud
doi: https://doi.org/10.1101/2021.03.18.21253903

Abstract

Objectives To develop and validate patient-reported instruments, based on patients’ lived experiences, for monitoring the symptoms and impact of long covid.

Design The long covid Symptom and Impact Tools (ST and IT) were constructed from the answers to a survey with open-ended questions to 492 patients with long covid. Validation of the tools involved adult patients with suspected or confirmed covid-19 and symptoms extending over three weeks after onset. Construct validity was assessed by examining the relations of the ST and IT scores with health related quality of life (EQ-5D-5L), function (PCFS, post-covid functional scale), and perceived health (MYMOP2). Reliability was determined by a test-retest. The “patient acceptable symptomatic state” (PASS) was determined by the percentile method.

Results Validation involved 1022 participants (55% with confirmed covid-19, 79% female and 12.5% hospitalised for covid-19). The long covid ST and IT scores were strongly correlated with the EQ-5D-5L (rs = −0.45 and rs = −0.59 respectively), the PCFS (rs = −0.39 and rs = −0.55), and the MYMOP2 (rs = −0.40 and rs = −0.59). Reproducibility was excellent with an interclass correlation coefficient of 0.83 (95% confidence interval 0.80 to 0.86) for the ST score and 0.84 (0.80 to 0.87) for the IT score. In total, 793 (77.5%) patients reported an unacceptable symptomatic state, thereby setting the PASS for the long covid IT score at 30 (28 to 33).

Conclusions The long covid ST and IT tools, constructed from patients’ lived experiences, provide the first validated and reliable instruments for monitoring the symptoms and impact of long covid.

Short summary We developed the long covid Symptom and Impact Tools (ST and IT) from the experiences of 492 patients, captured during a survey with open-ended questions, and assessed their validity and reliability in a sample of 1022 patients with long covid.

Introduction

As of February 2021, about 100 million people worldwide have been infected by the SARS-CoV 2 virus.1 According to some studies, from 10% to 70% of them, including many who only had a mild disease, report the presence of symptoms such as fatigue, dyspnoea, chest pain, memory problems, and concentration disorders persisting beyond three weeks from the initial onset of symptoms.2-8 Yet, so far, surveillance systems around the world remain focused exclusively on the acute aspect of the pandemic, tracking new cases, hospitalisations, and deaths, while neglecting these long-term consequences of covid-19.9

Persisting signs and symptoms three weeks after the acute covid-19 infection may be associated with several distinct clinical entities: 1) organ injury from the acute disease or its treatment (e.g., lung scarring, complications of intensive care, etc.),8,10-12 2) post-viral fatigue syndrome,13-15 3) prolonged viral shedding, especially in immunosuppressed patients,16 4) reinfection, and 5) mental factors, such as post-traumatic stress. From the patient’s perspective, all or several of these causes are often intertwined and produce a substantial burden of disease. As a result, patients themselves created the term “long covid” to describe all of the symptoms they could experience, independently of their specific cause.17

Today, advances in understanding and treating long covid are impeded by the heterogeneity of its assessment: each study uses different sets of symptoms, most generally defined from the care givers’ perspective.3,18,19 There is currently no comprehensive, validated, and reliable instrument for monitoring either the course of long covid symptoms or its impact on patients’ lives—an absence that incurs the risk of missing important patient perspectives. For example, many patients with long covid report an inability to sleep, difficulties concentrating, or exhaustion after mild exercise—manifestations not considered in all existing studies.20 The National Institute for Health and Care Excellence (NICE) recently called for the development and validation of scales to assess long covid,21 and in February 2021, the World Health Organization issued a standardised, albeit non-validated, case report form for the follow-up of patients after acute covid-19.22

In this study, we aimed to develop and validate a set of patient reported instruments for monitoring the symptoms and impact of long covid, usable in clinical practice and constructed from patients’ lived experience.

Methods

The development and validation of these instruments applied a two-stage method.23 The study was nested within ComPaRe (Communauté de Patients pour la Recherche, www.compare.aphp.fr), an e-cohort of patients with chronic conditions who volunteer to participate in research by regularly answering patient reported outcome measures (PROMs) and patient reported experience measures (PREMs).24 Because of the similarities between long covid and chronic conditions, recruitment in ComPaRe was extended to patients reporting symptoms of covid-19 lasting more than three weeks past initial onset. All participants provide electronic consent before participating. The Institutional Review Board of Hôtel-Dieu Hospital, Paris, approved ComPaRe (IRB: 0008367).

Stage 1: Development of a set of patient reported instruments to monitor the symptoms and impact of long covid from the patients’ lived experiences

We sought to develop the contents of the instruments from the patients’ lived experiences of long covid. We invited adult patients in the ComPaRe cohort who reported a covid-19 infection (laboratory confirmed or not) with symptoms persisting more than three weeks past the initial infection to complete an online survey of open-ended questions that asked them to describe in detail the symptoms, potential triggers, and the effects of long covid on their lives. The survey, which took place from October 14 to November 29, 2020, used broad questions to avoid directing patients about long covid’s consequences (Supplementary material 1).

Participants were recruited through a social media and media campaign and by partner patient associations. Participants who had participated were invited to encourage people in their entourage who may have had a COVID-19 infection, to participate, by a ‘snowball’ sampling method 25.

Two investigators (VTT and CR) used an inductive multiple-round qualitative content analysis to examine the participants’ responses. First, they independently coded the responses (i.e., they identified within each response any expression found in the text expressing a symptom or impact of long covid and assigned it a code). Answers related to the acute phase of covid-19 (that is, symptoms during the first three weeks of infection) were removed from analysis. For example, when a participant stated that “I had a cough and fever for one week and joint pain for 7 months”, we considered only “joint pain” to be a potential long covid symptom. If the response did not specify a timeframe, all reported symptoms were kept in the analysis. Next, the investigators grouped the identified codes into a standardised set of symptoms and consequences of long covid, based on their medical knowledge and the literature.4,18 Analysis continued on until a comprehensive list of symptoms was obtained, i.e., until data saturation was reached. The point of data saturation was assessed with a mathematical model to predict the number of new symptoms that could be identified by adding new participants to the study.26 In a third step, they reduced the number of symptoms by: 1) grouping closely related symptoms (eg, loss of taste and changes in taste were grouped as loss/change of taste) and 2) eliminating those expressed by less than 2% of participants in the open-ended survey.

The two investigators used the results of the open-ended survey to develop a preliminary patient reported instrument for monitoring long covid’s current symptoms and impact. We chose to split the instrument into two independent parts. The first part aimed at assessing patients’ long covid-related symptoms over the previous 30 days with the checklist of symptoms identified above, a list we named the long covid Symptom Tool (ST). The second part aimed at assessing the disease’s impact on their lives over the past 30 days from the responses to six questions constructed from themes identified and examples provided by patients in their open-ended answers. Items used a numeric scale ranging from 0 (no impact) to 10 (maximal impact). This second instrument was called the long covid Impact Tool (IT). These two instruments were reappraised by the two author patients (BC and CC) for content validity, clarity, and wording during a telephone interview with the main investigator (VTT) that used the double interview method.27

Stage 2: Validation of the long covid symptom and impact tools

A second sample of adult patients reporting a SARS-CoV 2 infection (laboratory confirmed or not) with symptoms persisting three weeks past the initial infection served as the validation sample. Patients who had participated in the first stage could also participate in the validation stage. In addition, we increased our sample through a call for participation on the “TousAntiCOVID” app, the official French contact tracing app used by 12 million people. As our instrument intends to evaluate patients’ current symptoms and their impact, only patients reporting at least one symptom during the previous 30 days could participate in the validation study.

Each tool was validated independently. The long covid ST score was defined as the number of symptoms reported by patients, and the long covid IT score as the sum of the responses to each impact question.

Construct validity was determined by confirming several theories or conceptions (ie, constructs) about long covid. First, we hypothesised that both the ST and IT scores would be negatively correlated with patients’ quality of life, which was evaluated with the EuroQol five-dimension five-level (EQ-5D-5L) questionnaire and the EuroQol visual analogue scale (EQ-VAS).28 Our second hypothesis was that both the ST and IT scores would be positively correlated with patients’ functional status after covid-19, assessed by the Post-Covid Functional Scale (PCFS), an ordinal tool categorising patients in 5 grades ranging from 0 “No functional limitation” to 4 “Severe functional limitations”.29,30 Finally, we hypothesised that both the ST and IT scores would also be negatively correlated with patients’ general perceived health, assessed with the Measure Yourself Medical Outcome Profile 2 (MYMOP2) tool, an instrument focused on complaints that patients identified as most important to them.31 Correlations were assessed by Spearman correlation coefficients (rs) and considered high with rs > 0.50 and moderate with rs = 0.35 to 0.50.

The reliability of both the ST and IT scores was determined by the test-retest method. Some patients completed the questionnaires twice: at baseline and two weeks later. Reproducibility was assessed by the intraclass correlation coefficient (ICC) for agreement.32 The 95% confidence intervals (95% CIs) were determined by a bootstrap method. Reproducibility was considered acceptable with ICC > 0.70.33 Reliability was also tested by Bland and Altman plots, which present the differences between two measurements against the means of the two measurements.34 Finally, internal consistency was assessed by Cronbach’s α and was considered acceptable between 0.70 and 0.95.35

The Patient Acceptable Symptom State (PASS) is the level of a continuous treatment outcome measure below which patients consider themselves well.36 The PASS for the long covid IT was determined by matching the scores to an anchor question: “Taking into account all your symptoms in daily life and your functional impairment, do you consider that your current state is satisfactory?” The threshold was the impact score below which 75% of patients considered their symptom state acceptable.37 Percentile bootstrap with 2000 replications provided the 95% CIs.

Statistical analyses were performed with R v. 3.6.3.

Patient and public involvement

The tools were developed from qualitative data about patients’ lived experiences, captured during a survey with open-ended questions. The qualitative results and the implementation of the items in the patient reported tools were discussed in detail with two patients meeting the criteria for long covid. They also participated in the critical revision of the manuscript and are co-authors of the paper (BC and CC).

Results

Step 1: Development of the long covid symptom and impact tools from patients’ lived experiences

During the survey period (from October 14 to November 29, 2020), 528 members of the ComPaRe cohort received emails inviting them to complete the open-ended survey, and 492 (93%) did so. Their median age was 45 years (IQR 35 to 50.25), and 414 (84%) were women. In total, 210 (43%) had tested positive for SARS-CoV2 by PCR swab or serological assay and were considered confirmed covid-19 patients. Median time since symptom onset was 217 days (IQR 205 to 230) (Supplementary material 2).

Participants’ open-ended answers represented a corpus of 116 657 words. Inductive content analysis of patients’ answers was stopped after the analysis of 380 answers and the definition of 53 symptoms of long covid because models showed that 99.99% of all possible symptoms had been identified in both the total sample and in the subgroups of confirmed and suspected cases (Supplementary material 3). The list of symptoms was further organised in 10 categories: general symptoms (n=11), thorax (n=6), digestive (n=3), ear/nose/throat (n=5), eyes (n=3), genitourinary (n=2), hair and skin (n=4), musculoskeletal (n=4), neurological (n=11), and blood and lymph circulation (n=4) symptoms (Supplementary material 4).

The analysis of patients’ open-ended answers also enabled us to identify six aspects of patients’ lives that were affected by long covid: 1) difficulties in performing personal activities such as driving, shopping, or doing household chores; 2) difficulties in their professional lives; 3) difficulties in fulfilling their family roles and/or feeling that they’re a burden to their family or friends; 4) difficulties related to social activities; 5) morale, fear of the future and of not returning to normal, and 6) negative impact on relationships with care providers. Examples of what patients wrote are provided in Supplementary material 5.

Thus the list of symptoms and the themes and examples related to the impact of long covid on patients’ lives were used to develop preliminary versions of the long covid tools, which were further reviewed and improved by two patients. The Box presents the items for both tools.

Step 2: Validation of the long covid symptom and impact tools

From November 30, 2020, to February 15, 2021, 1360 ComPaRe participants reported they had had long covid. In all, 1022 had experienced at least one covid-19-related symptom in the 30 days before responding to the survey and were included in the validation sample. Among them, 418 (40%) had participated in the development sample, in the first stage. Table 1 presents the characteristics of all patients included in the validation sample. Their median age was 45 years (IQR 37 to 52), and 817 (79.9%) were women. Overall, 564 (55%) had tested positive for SARS-CoV2 by PCR swab or serological assay and were considered confirmed cases. Among all cases, 128 (12.5%) patients had been hospitalised for covid-19, and 17 (1.7%) had been admitted to an ICU. The median time since symptom onset was 263 days (IQR 123 to 289), with a bimodal distribution corresponding to the first (260 to 300 days) and second waves (50 to 100 days) in France.

View this table:
Table 1: Characteristics of patients included in the validation step (n=1022)

Symptoms and impact of long covid

The median long covid ST score was 16 (IQR 11 to 23). This score reports the number of symptoms patients experienced over the last 30 days and has a theoretical range from 0 (no symptoms) to 53 (all symptoms identified during step 1). The symptoms most frequently reported were fatigue (n=899), headaches (n=709), difficulties concentrating/mental fog (n=650), sleep disorders (n=603), and dyspnoea (n=570) (Figure 1). Symptom frequency was similar in confirmed and suspected cases except for “change/loss of taste” and “change/loss of smell”, both more frequent in confirmed cases. Besides symptoms, 824 patients (80.6%) reported a relapsing-remitting disease course with daily (n=285, 34.6%), weekly (n=320, 38.8%), or less than weekly (n=219, 26.6%) relapses.

Figure 1:
Figure 1: Symptoms reported by patients during the validation step (n=1022)

Patients’ median long covid IT score was 36 (IQR 24 to 45). This score has a theoretical range of 0 (no impact) to 60 (maximum impact) and represents the sum of item scores for the six questions related to the disease’s impact on their personal activities, family lives, professional lives, social lives, their morale, and their relationships with care providers. In all, 265 (26%) patients rated the impact of the disease on their work lives at 10 (out of 10). In general, the disease’s impact on patients’ lives was similar for patients with confirmed and suspected infections, except for the item about its effect on their relationships with care givers, which was rated higher in suspected cases (Supplementary material 6).

The long covid ST and IT scores were highly correlated (rs=0.54, p<0.0001) and did not seem to differ by time from symptom onset (Supplementary material 7).

Construct validity

Overall, 970 (95%) patients completed the EQ-5D-5L quality of life questionnaire, the EQ-VAS scale, and the PCFS functional status scale, while 746 (73%) answered the MYMOP2 questionnaire.

For health-related quality of life, the median EQ-5D-5L and EQ-VAS values in our sample were respectively 0.83 (IQR 0.60 to 0.89) and 51 (IQR 40 to 70). As hypothesised, we found that the long covid ST score was moderately and negatively correlated with the EQ-5D-5L questionnaire (rs = −0.45, p<0.0001) and the EQ-VAS (rs = −0.39, p<0.0001), while the long covid IT score had a strongly negative correlation with the EQ-5D-5L questionnaire (rs = − 0.59, p<0.0001) and the EQ-VAS (rs = −0.54, p<0.0001).

For functional assessments, 469 (48%) patients in our sample indicated that they were no longer able to perform some activities at home or at work by themselves (grade 3 or 4 of the PCFS), and 371 (38%) reported that they had had to reduce some of their activities (grade 2 of the PCFS). We found a moderate correlation between the long covid ST score and the PCFS score (rs = −0.39, p<0.0001) and a high correlation between the long covid IT score and the PCFS score (rs = −0.55, p<0.0001).

Similarly, for patients’ perceived health state, we found that the long covid ST score was moderately correlated with the MYMOP2 score (rs = −0.40, p<0.0001) while the long covid IT score was highly correlated with it (rs = −0.59, p<0.0001) (Table 2).

View this table:
Table 2: Relations between patients’ long covid symptom tool (ST) score, impact tool (IT) score, quality of life, functional status, and perceived health (n=970)

Reliability

Of the 351 patients invited to complete the long covid ST and IT twice for the test-retest, 235 (67%) did so. The symptom score had an ICC of 0.83 (95% CI 0.80 to 0.86), with Bland and Altman plots showed a mean difference of 0.8 (95% limits of agreement, −14 and 16). The impact score’s ICC was 0.84 (95% CI 0.80 to 0.87), with Bland and Altman plots showing a mean difference of 0.5 (95% limits of agreement, −11 to 12 (Figure 2). Finally, Cronbach’s alpha was 0.89 (95% CI, 0.88 to 0.90) for the long covid ST score and 0.86 (95% CI, 0.85 to 0.88) for the IT score.

Figure 2:
Figure 2: Bland and Altman plots for the test-retest of the long covid Symptom Tool score (A) and Impact Tool Score (B) scores (n=235)

Patient acceptable symptom state

Only 229 (22.4%) patients reported an acceptable symptomatic state, while 793 (77.5%) patients reported it was unacceptable. Long covid’s impact was considered acceptable for >75% of patients with IT scores < 30 (95% CI, 28 to 33), that is, below 50% of the maximum score. (Figure 3).

Figure 3:
Figure 3: Empirical cumulative distribution function for long covid Impact Tool score, in patients considering their levels of symptoms and impact acceptable (n=229) and not acceptable (n=793)

Discussion

In this study, we present the long covid ST and IT. They are the first set of validated and reliable instruments for monitoring this disease’s symptoms and impact and assess respectively 53 long covid symptoms and 6 dimensions of patients’ lives that it can affect.

Until now, two issues have limited the assessment of long covid. First, the sets of symptoms used have been both unreliable and defined from the care givers’ perspective. The number of symptoms assessed in studies has ranged from 8 to 205 manifestations, the latter also including triggers and laboratory test result anomalies3,7,8,19,38 and usually failed to cover difficulties in concentration, even though several studies report that this symptom is frequent in long covid.8,38 The long covid ST and IT were built from patients’ lived experiences, captured during a survey with open-ended questions. Our tools thus provide a comprehensive picture of the disease. By using mathematical models to assess data saturation, we ensured that our instruments covered all manifestations and impacts of long covid relevant to patients. Both of our tools showed robust psychometric properties. Construct validity was demonstrated by the high correlations with patients’ quality of life, functional status, and perceived health state. Reliability was excellent with an ICC ≥ 0.8 during the test-retest. Finally, we defined a PASS (Patient Acceptable Symptomatic State) for long covid that provides thresholds to be met by pharmacological and/or non-pharmacological interventions aimed at reducing its impact on patients’ lives.

Second, most studies have focused on counting symptoms, but have failed to assess the disease’s impact on patients’ lives.4,5,7,18,19 The long covid IT fills this gap by providing validated, and reliable questions for assessing its burden. This is critical, in view of the major impact that long covid has on the quality of life and functioning of the patients in our ComPaRe cohort. The health related quality of life for patients with long covid, assessed with the EQ-VAS, was on average 40% lower than in the reference general population,39 and similar to that of patients living with epilepsy or multiple sclerosis.40 Half reported impaired functioning, that is, they were no longer able to perform some activities unassisted at home or at work (PCFS grade 3 or 4), a figure similar to that described in the study by Davis et al.38

The symptoms captured in the long covid ST overlap with those presented in sections 2.5 and 2.6 of the recent case report form issued by the WHO for the follow-up of patients after their acute illness. The WHO form covers 45 symptoms of long Covid (all included in our instrument) and includes questions on patients’ functioning (e.g., washing themselves, dressing, ability to join in community activities, etc.).22 Our set of instruments goes further by inquiring about patients’ perceptions of the disease’s impact on their lives, roles, and relationships, beyond assessing whether they can or cannot perform specific activities. Moreover, besides collecting information, our instruments provide a scoring method producing valid and reproducible measurements of the disease- and patient-relevant cut-offs to interpret these measures. In all, the overlap of the long covid ST and IT with the WHO case report form strengthens the content validity of our instruments and provides a glimpse of the potential validity and reliability of the WHO form.

This study has limitations. Generalisation of the estimates obtained in this study must be cautious. Our study recruited volunteer patients who reported persistent symptoms. This might have selected younger, more educated, and more often female patients willing to share their experiences with others and/or with more severe conditions. Nonetheless, we were able to involve and validate our instrument in a diverse sample of participants by using a broad media campaign and a call for participation on the “TousAntiCOVID” app.41 Second, our study may not be appropriate for examining the longitudinal impact of the disease. Although we recruited patients with various times from symptom onset, the assessment of symptoms and impact was cross-sectional. Future work following up patients with standardised, validated tools is required to investigate the duration of the disease and the course of the symptoms over time. Finally, in view of the limited number of patients who were hospitalised in ICUs in our study, we cannot confirm that our tool is suitable for measuring the consequences of ICU on patients.

According to the United Kingdom Office for National Statistics, 10% of people who were infected with SARS-CoV-2 still experience symptoms after three months, including those whose acute infection was asymptomatic.42 With about 100 million cases of COVID-19 worldwide, long covid may well affect millions of patients. The severe burden of illness and the impairment of quality of life associated with long covid call for urgent research to understand this disease and to develop interventions to help patients. Using scientific, valid, and reliable measurements in these initiatives will enable the comparison and combination of study results.

Conclusions

The long covid Symptom and Impact Tools, constructed from patients’ lived experience, provide the first validated, reliable instruments for monitoring the symptoms and impact of long covid. It may help the development of treatment strategies to mitigate the considerable burden of this disease.

Data Availability

All data collected for the study, including individual participant data and a data dictionary is available for research purposes, under the rules of the ComPaRe e-cohort (https://compare.aphp.fr/)

Contributions

Generated the idea: VTT and PR, Conceived and designed the experiments: VTT, and PR, Collected data: VTT, YY and PR. Analysed data: VTT, CR, BC, CC; Wrote the first draft of the manuscript: VTT, Contributed to the writing of the manuscript: VTT, CR, BC, CC, YY and PR; ICMJE criteria for authorship read and met: VTT, CR, BC, CC, YY and PR. Agree with manuscript results and conclusions: VTT, CR, BC, CC, YY and PR. VTT is the guarantor, had full access to the data in the study, and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Transparency statement

VTT affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted. There were no discrepancies from the study as originally planned.

Dissemination to participants and related patient and public communities

The results of your study will be sent to research participants and partner patients’ associations, accordingly to the protocol of ComPaRe. In addition, we plan a press release for the study.

Financial disclosure

The authors received no specific funding for this work.

Declaration of interests

The authors declare no competing interests and no financial associations that may be relevant or seen as relevant to the submitted manuscript.

The authors have no association with commercial entities that could be viewed as having an interest in the general area of the submitted manuscript.

Data Sharing

All data collected for the study, including individual participant data and a data dictionary is available for research purposes, under the rules of the ComPaRe e-cohort (https://compare.aphp.fr/)

List of Supplemental files

  • Supplementary material 1: Open-ended questions used for the development of the tool (step 1)

  • Supplementary material 2: Characteristics of patients included in the survey with open-ended questions (step 1) (n=492)

  • Supplementary material 3: Data saturation for symptoms identified during the survey with open-ended questions

  • Supplementary material 4: Symptoms identified in the survey with open-ended questions (n=380)

  • Supplementary material 5: Examples of what patients wrote about the impact of long Covid on their lives (n=380)

  • Supplementary material 6: Patients’ scores (median [interquartile range]) to the items assessing the impact of long Covid (n=1022)

  • Supplementary material 8: Long covid Symptom Tool (ST) score, Impact Tool (IT) score and EQ-5D-5L scores over time (n=970) (n=970)

Box:
Box:
Box: Items of the long covid symptom and impact tools (ST and IT)

Acknowledgements

The authors thank Elise Diard and Isabelle Pane for their help in the survey development, Isabelle Pane for data management and Elise Diard for drafting the figures.

References

  1. 1.↵
    World Health Organization. WHO Coronavirus Disease (COVID-19) Dashboard: World Health organization; 2020 [Available from: https://covid19.who.int/ accessed 06/11 2020.
  2. 2.↵
    Garrigues E, Janvier P, Kherabi Y, et al. Post-discharge persistent symptoms and health-related quality of life after hospitalization for COVID-19. J Infect 2020 doi: 10.1016/j.jinf.2020.08.029 [published Online First: 2020/08/28]
    OpenUrlCrossRefPubMed
  3. 3.↵
    Carfí A, Bernabei R, Landi F. Persistent Symptoms in Patients After Acute COVID-19. Jama 2020;324(6):603–05. doi: 10.1001/jama.2020.12603 [published Online First: 2020/07/10]
    OpenUrlCrossRefPubMed
  4. 4.↵
    Singh SM, Reddy C. An Analysis of Self-reported Longcovid Symptoms on Twitter. medRxiv 2020:2020.08.14.20175059. doi: 10.1101/2020.08.14.20175059
    OpenUrlAbstract/FREE Full Text
  5. 5.↵
    Tenforde MW, Kim SS, Lindsell CJ, et al. Symptom Duration and Risk Factors for Delayed Return to Usual Health Among Outpatients with COVID-19 in a Multistate Health Care Systems Network - United States, March-June 2020. MMWR Morbidity and mortality weekly report 2020;69(30):993–98. doi: 10.15585/mmwr.mm6930e1 [published Online First: 2020/07/31]
    OpenUrlCrossRefPubMed
  6. 6.
    Greenhalgh T, Knight M, A’Court C, et al. Management of post-acute covid-19 in primary care. BMJ (Clinical research ed) 2020;370:m3026. doi: 10.1136/bmj.m3026 [published Online First: 2020/08/14]
    OpenUrlFREE Full Text
  7. 7.↵
    Nehme M, Braillard O, Alcoba G, et al. COVID-19 Symptoms: Longitudinal Evolution and Persistence in Outpatient Settings. Annals of internal medicine 2020 doi: 10.7326/m20-5926 [published Online First: 2020/12/08]
    OpenUrlCrossRef
  8. 8.↵
    Huang C, Huang L, Wang Y, et al. 6-month consequences of COVID-19 in patients discharged from hospital: a cohort study. Lancet (London, England) 2021;397(10270):220–32. doi: 10.1016/s0140-6736(20)32656-8 [published Online First: 2021/01/12]
    OpenUrlCrossRef
  9. 9.↵
    Alwan NA. Surveillance is underestimating the burden of the COVID-19 pandemic. Lancet (London, England) 2020;396(10252):e24. doi: 10.1016/s0140-6736(20)31823-7 [published Online First: 2020/08/31]
    OpenUrlCrossRef
  10. 10.↵
    Puntmann VO, Carerj ML, Wieters I, et al. Outcomes of Cardiovascular Magnetic Resonance Imaging in Patients Recently Recovered From Coronavirus Disease 2019 (COVID-19). JAMA cardiology 2020;5(11):1265–73. doi: 10.1001/jamacardio.2020.3557 [published Online First: 2020/07/31]
    OpenUrlCrossRefPubMed
  11. 11.
    Han X, Fan Y, Alwalid O, et al. Six-Month Follow-up Chest CT findings after Severe COVID-19 Pneumonia. Radiology 2021:203153. doi: 10.1148/radiol.2021203153 [published Online First: 2021/01/27]
    OpenUrlCrossRef
  12. 12.↵
    Nersesjan V, Amiri M, Lebech AM, et al. Central and peripheral nervous system complications of COVID-19: a prospective tertiary center cohort with 3-month follow-up. Journal of neurology 2021:1–19. doi: 10.1007/s00415-020-10380-x [published Online First: 2021/01/14]
    OpenUrlCrossRef
  13. 13.↵
    Lam MH, Wing YK, Yu MW, et al. Mental morbidities and chronic fatigue in severe acute respiratory syndrome survivors: long-term follow-up. Arch Intern Med 2009;169(22):2142–7. doi: 10.1001/archinternmed.2009.384 [published Online First: 2009/12/17]
    OpenUrlCrossRefPubMed
  14. 14.
    Townsend L, Dyer AH, Jones K, et al. Persistent fatigue following SARS-CoV-2 infection is common and independent of severity of initial infection. PloS one 2020;15(11):e0240784. doi: 10.1371/journal.pone.0240784 [published Online First: 2020/11/10]
    OpenUrlCrossRefPubMed
  15. 15.↵
    Hickie I, Davenport T, Wakefield D, et al. Post-infective and chronic fatigue syndromes precipitated by viral and non-viral pathogens: prospective cohort study. BMJ (Clinical research ed) 2006;333(7568):575. doi: 10.1136/bmj.38933.585764.AE [published Online First: 2006/09/05]
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    Aydillo T, Gonzalez-Reiche AS, Aslam S, et al. Shedding of Viable SARS-CoV-2 after Immunosuppressive Therapy for Cancer. The New England journal of medicine 2020;383(26):2586–88. doi: 10.1056/NEJMc2031670 [published Online First: 2020/12/02]
    OpenUrlCrossRefPubMed
  17. 17.↵
    Callard F, Perego E. How and why patients made Long Covid. Soc Sci Med 2021;268:113426. doi: 10.1016/j.socscimed.2020.113426 [published Online First: 2020/11/18]
    OpenUrlCrossRef
  18. 18.↵
    Sudre CH, Murray B, Varsavsky T, et al. Attributes and predictors of Long-COVID: analysis of COVID cases and their symptoms collected by the Covid Symptoms Study App. medRxiv 2020:2020.10.19.20214494. doi: 10.1101/2020.10.19.20214494
    OpenUrlAbstract/FREE Full Text
  19. 19.↵
    Carvalho-Schneider C, Laurent E, Lemaignen A, et al. Follow-up of adults with noncritical COVID-19 two months after symptom onset. Clin Microbiol Infect 2020 doi: 10.1016/j.cmi.2020.09.052 [published Online First: 2020/10/09]
    OpenUrlCrossRef
  20. 20.↵
    Long COVID: let patients help define long-lasting COVID symptoms. Nature 2020;586(7828):170. doi: 10.1038/d41586-020-02796-2 [published Online First: 2020/10/09]
    OpenUrlCrossRef
  21. 21.↵
    NICE. COVID-19 rapid guideline: managing the long-term effects of COVID-19: NICE, 2020.
  22. 22.↵
    World Health Organization. Global COVID-19 Clinical Platform Case Report Form (CRF) for Post COVID condition (Post COVID-19 CRF): World Health Organization,; 2021 [Available from: https://www.who.int/publications/i/item/global-covid-19-clinical-platform-case-report-form-(crf)-for-post-covid-conditions-(post-covid-19-crf-) accessed 18/02/2021.
  23. 23.↵
    Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. Journal of clinical epidemiology 2007;60(1):34–42. doi: 10.1016/j.jclinepi.2006.03.012 [published Online First: 2006/12/13]
    OpenUrlCrossRefPubMedWeb of Science
  24. 24.↵
    Tran VT, Ravaud P. COllaborative open platform E-cohorts for research acceleration in trials and epidemiology. sJournal of clinical epidemiology 2020;124:139–48. doi: 10.1016/j.jclinepi.2020.04.021 [published Online First: 2020/05/08]
    OpenUrlCrossRef
  25. 25.↵
    Denscombe M, editor. The Good Research Guide. Buckingham: Open University Press, 1997.
  26. 26.↵
    Tran VT, Porcher R, Tran VC, et al. Predicting data saturation in qualitative surveys with mathematical models from ecological research. Journal of clinical epidemiology 2017;82:71–78 e2. doi: 10.1016/j.jclinepi.2016.10.001 [published Online First: 2016/10/30]
    OpenUrlCrossRef
  27. 27.↵
    Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne 1986;134(8):889–95. [published Online First: 1986/04/15]
    OpenUrl
  28. 28.↵
    Johnson JA, Coons SJ, Ergo A, et al. Valuation of EuroQOL (EQ-5D) health states in an adult US sample. PharmacoEconomics 1998;13(4):421–33. [published Online First: 1998/03/08]
    OpenUrlCrossRefPubMedWeb of Science
  29. 29.↵
    Klok FA, Boon G, Barco S, et al. The Post-COVID-19 Functional Status scale: a tool to measure functional status over time after COVID-19. Eur Respir J 2020;56(1) doi: 10.1183/13993003.01494-2020 [published Online First: 2020/05/14]
    OpenUrlAbstract/FREE Full Text
  30. 30.↵
    Machado FVC, Meys R, Delbressine JM, et al. Construct validity of the Post-COVID-19 Functional Status Scale in adult subjects with COVID-19. Health Qual Life Outcomes 2021;19(1):40. doi: 10.1186/s12955-021-01691-2 [published Online First: 2021/02/05]
    OpenUrlCrossRef
  31. 31.↵
    Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ (Clinical research ed) 1996;312(7037):1016–20. [published Online First: 1996/04/20]
    OpenUrlAbstract/FREE Full Text
  32. 32.↵
    Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin 1979;86(2):420–8. doi: 10.1037//0033-2909.86.2.420 [published Online First: 1979/03/01]
    OpenUrlCrossRefPubMedWeb of Science
  33. 33.↵
    Streiner D, Norman G. Reliability. Health Measurement Scales: a practical guide to their development and use. Fourth Edition ed. Oxford: Oxford University Press 2008.
  34. 34.↵
    Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet (London, England) 1986;1(8476):307–10. [published Online First: 1986/02/08]
    OpenUrl
  35. 35.↵
    Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16(3)
  36. 36.↵
    Tubach F, Dougados M, Falissard B, et al. Feeling good rather than feeling better matters more to patients. Arthritis and rheumatism 2006;55(4):526–30. doi: 10.1002/art.22110 [published Online First: 2006/07/29]
    OpenUrlCrossRefPubMedWeb of Science
  37. 37.↵
    Tubach F, Ravaud P, Baron G, et al. Evaluation of clinically relevant states in patient reported outcomes in knee and hip osteoarthritis: the patient acceptable symptom state. Ann Rheum Dis 2005;64(1):34–7. doi: 10.1136/ard.2004.023028 [published Online First: 2004/05/08]
    OpenUrlAbstract/FREE Full Text
  38. 38.↵
    Davis H, Assaf G, McCorkell L, et al. Characterizing Long COVID in an International Cohort: 7 Months of Symptoms and Their Impact. medRxiv 2020 doi: https://doi.org/10.1101/2020.12.24.20248802
  39. 39.↵
    Perneger TV, Combescure C, Courvoisier DS. General population reference values for the French version of the EuroQol EQ-5D health utility instrument. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2010;13(5):631–5. doi: 10.1111/j.1524-4733.2010.00727.x [published Online First: 2010/04/24]
    OpenUrlCrossRefPubMedWeb of Science
  40. 40.↵
    Prevolnik Rupel V, Divjak M, Zrubka Z, et al. EQ-5D studies in nervous system diseases in eight Central and East European countries: a systematic literature review. The European journal of health economics : HEPAC : health economics in prevention and care 2019;20(Suppl 1):109–17. doi: 10.1007/s10198-019-01068-9 [published Online First: 2019/05/18]
    OpenUrlCrossRef
  41. 41.↵
    Kesse-Guyot E, Andreeva V, Castetbon K, et al. Participant profiles according to recruitment source in a large Web-based prospective study: experience from the Nutrinet-Sante study. Journal of medical Internet research 2013;15(9):e205. doi: 10.2196/jmir.2488 [published Online First: 2013/09/17]
    OpenUrlCrossRefPubMed
  42. 42.↵
    Office for National Statistics. The prevalence of long COVID symptoms and COVID-19 complications. London, UK, 2020.
Back to top
Posted March 20, 2021.
Download PDF

Supplementary Material

Data/Code
Email
Development and validation of the long covid symptom and impact tools, a set of patient-reported instruments constructed from patients’ lived experience
Viet-Thi Tran, Caroline Riveros, Bérangère Clepier, Moïse Desvarieux, Camille Collet, Youri Yordanov, Philippe Ravaud
medRxiv 2021.03.18.21253903; doi: https://doi.org/10.1101/2021.03.18.21253903
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area