Information given by websites selling home self-sampling COVID-19 tests: An analysis of accuracy and completeness

S Taylor-Phillips, S Berhane, AJ Sitch, K Freeman, MJ Price, C Davenport, J Geppert, IM Harris, O Osokogu, M Skrybant, JJ Deeks
doi: https://doi.org/10.1101/2020.08.18.20177360

ABSTRACT

Objectives To assess the accuracy and completeness of information provided by websites selling home self-sampling and testing kits for COVID-19.

Design Cross-sectional observational study.

Setting All websites (n = 27) selling direct to user home self-sampling and testing for COVID-19 (41 tests) in the UK (39 tests) and US (2 tests) identified by a website search on 23rd May 2020.

Main outcome measures Thirteen predefined basic information items to communicate to a user, including who should be tested, when and how testing should be done, test accuracy, and interpretation of results.

Results Many websites did not provide the name or manufacturer of the test (32/41; 78%), when to use the test (10/41; 24%), test accuracy (12/41; 29%), and how to interpret results (21/41; 51%). Sensitivity and specificity were the most commonly reported test accuracy measures (either reported for 27/41 (66%) tests); we could only link these figures to manufacturers’ documents or publications for four (10%) tests. Predictive values, most relevant to users, were rarely reported (five [12%] tests reported positive predictive values). For molecular virus tests, 9/23 (39%) websites explained that test positives should self-isolate, and 8/23 (35%) explained that test negatives may still have the disease. For antibody tests, 12/18 (67%) websites explained that testing positive does not necessarily infer immunity from future infection. Seven (39%) websites selling antibody tests claimed the test had a CE mark, when they were for a different intended use (venous blood rather than finger-prick samples).

Conclusions At the point of online purchase of home self-sampling COVID-19 tests, users in the UK are provided with incomplete, and in some cases misleading information on test accuracy, intended use and test interpretation. Best practice guidance for communication about tests to the public should be developed and enforced for online sales of COVID-19 tests.

Strengths and Weaknesses

  • We believe this is the first research on accuracy of information provided by websites selling tests for COVID-19, where users may put themselves or others at increased risk of transmission if results are misinterpreted.

  • We duplicated processes of searching and data extraction to minimise bias

  • Using pre-specified criteria, we found evidence that websites selling home self-sampling COVID-19 tests provided incomplete and inaccurate information on test accuracy and interpretation of test results at the point of purchase.

  • We developed basic guidance on what should be communicated when selling tests, including the type of test; situations when the test should be used; the time when the test should be done and details of how it should be done; the name of the test and details from clinical accuracy studies; evidence of compliance with regulatory approvals; explanation of test results using accessible and relevant metrics such as predictive values; and guidance to the interpretation and actions based on results.

  • We only included websites from the UK and US, so whilst the principles of what should be communicated apply to all countries, the results about data completeness are not generalisable beyond the UK and US.

INTRODUCTION

The 2019 novel coronavirus (COVID-19) pandemic has resulted in national population measures such as restricted movement (‘lockdown’), and mass testing programmes. Testing is regarded as critical to manage the pandemic – the two main test types available being molecular virus tests (to detect current infection) and antibody tests (to detect previous infection). The World Health Organization (WHO) recommends polymerase chain reaction (PCR)-based molecular virus testing of symptomatic individuals to detect current COVID-19 infection,1 to enable identification and isolation of confirmed cases, and tracing of those exposed for further testing. However, due to the sensitivity for a single PCR test being as low as 70%,2 the WHO states that even two consecutive negative PCR tests do not rule out infection with COVID-19.1 Antibody tests are not recommended for individual use by the WHO, because we do not yet understand whether presence of antibodies infers immunity from future infection. Their sensitivity has been estimated at around 80-90%, thus there is also a risk of false negatives.3 Timing of testing is critical for both tests: molecular tests are thought to be most accurate when used within five days of the onset of symptoms,4 antibody tests are most accurate two or more weeks after onset of symptoms.3

There are now multiple websites selling both molecular virus tests and antibody tests outside of national testing programmes. To ensure appropriate use, interpretation and actions following testing, it is necessary for tests to be sold with clear communication about who should use each test, when and how samples should be taken, and the implications of positive and negative results. Previous research investigating direct to user sales of genetic testing found that the information provided was incomplete, particularly the implications of test results and limitations of testing, and was not always in an accessible and understandable format.5–7

Direct to user sale of tests are regulated by the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK and the Food and Drug Administration (FDA) in the US. Europe8 and the USA9 operate a risk based regulation for in vitro diagnostic devices (IVDs) which depends on the intended use of the test and indications for use. IVDs for home testing fall into higher risk categories reflecting the fact they are initiated, performed and interpreted without professional guidance and require evidence that lay users correctly use the test and understand test results. Lay user studies are required as the basis for the instructions for use (IFU) document for the IVD.10 Home sampling tests are different from home testing as they receive approval based on home collected specimens with the test analysis being undertaken by professionals. At the time of writing, there were no COVID-19 antibody tests with a CE mark for either home sampling or home testing11 (the two COVID-19 antibody tests purchased by the UK Government are approved for use in venous but not finger-prick blood samples) whilst several molecular virus tests have regulatory approval for home sampling and are being used in the UK track and trace programme.12 Most websites selling COVID-19 tests would be classified by the MHRA as ‘distributors’, which gives clear obligations to supply the information provided by manufacturers with the test, but no specific guidance around communication on the website at the point of sale. Such claims are covered by the Advertising Standards Agency. In the US, there are no COVID-19 antibody tests with regulatory approval for home testing but four molecular (PCR) virus tests that have approval for home sampling13 where the appropriateness of the test purchase is assessed by a professional either pre-purchase or following a purchase request.

We analysed the information given to individuals considering purchasing a molecular virus or antibody COVID-19 test online for home self-sampling. We chose to review tests for sale in both the UK and US to cover two different regulatory systems with contrasting health services. We recorded information regarding who should be tested and when, claims about test accuracy, and information about how to interpret results. As the MHRA instigated a withdrawal of sales of antibody tests based on finger-prick blood samples on May 29th 2020 where tests require venous blood samples,11 we also evaluated how test vendors have responded.

METHODS

Our research question was how complete, accurate and informative is the information that online websites selling home self-sampling and testing for COVID-19 provide to the public?

Identification of websites

The search was designed to identify a representative sample of websites and online advertisements which would be seen by an individual searching for a non-specific COVID-19 test. We aimed to identify websites selling home self-sampling and testing for COVID-19 using molecular virus and/or antibody tests directly to users. Two researchers performed the searches independently on the same day (23rd May 2020) using the Google search engine in incognito mode in Google Chrome, with geo-locations for the UK and for the USA. In order to emulate a simple search for a non-specific coronavirus test, the search terms were (coronavirus OR covid-19 OR covid19) AND (test OR testing OR kit). Two researchers independently screened all results against the inclusion criteria, disagreements were resolved by a third researcher. For the UK search, we included websites moved to the top of the search results through advertisements, in order to mirror what a user would have seen on that day.

Inclusion criteria

We included websites selling molecular virus and/or antibody tests for COVID-19 direct to users in either the US or UK. We included point-of-care and laboratory-based tests, with the proviso that the sample was taken at home by the individual themselves. We excluded tests with assisted sampling (e.g. drive-through testing), or where part of the testing process before purchase included video, telephone or in-person contact with a medical professional (as we could not objectively assess the information content of such interactions). We included websites selling tests both via direct purchase and insurance funding, but excluded local or national government websites providing tests (including Public Health England [PHE] and the UK National Health Service [NHS]), and websites providing tests as part of a research study. We included all eligible tests, including where a single website sold multiple eligible tests. We excluded websites with a minimum order of more than a single test, as these targeted suppliers rather than individual users.

Data extraction

We extracted information about the test manufacturer and type of test; when testing was recommended; claims made about test accuracy; the advice given about changing behaviour in light of test results; accreditation; and the test cost. We assessed the information provided against a predefined list of items which we would expect to be communicated to a person considering purchasing a test for COVID-19, detailed in Table 1.

View this table:
Table 1:

Predefined information items which we would expect to be communicated to a person considering purchasing a test for COVID-19, and misinformation items which we would consider inappropriate to communicate, with rationale.

We extracted claims made about regulatory approval of the tests, in particular CE-IVD approval in the UK and FDA approval in the US, and where possible compared claims to the actual approval status for the test. We also extracted claims made about approval from non-regulatory bodies such as PHE and the NHS.

Website contents were extracted between 23rd and 28th May 2020. One researcher extracted data from each website onto a predefined data extraction form, and downloaded the website as an image file. A second researcher checked each extraction using the pdf copy to exclude temporal changes.

Patient and public involvement

A public contributor (MS), with both experience of being involved in research and leading public involvement in research, provided input into this project. MS has an interest in communicating scientific information to lay audiences. The rapid timeframes in which the research was conducted limited the scope for more comprehensive public involvement. MS contributed to discussions, paper drafts and is included as a co-author.

Ethics approval was not required for this review of publicly available documents.

RESULTS

For the UK our Google searches retrieved 550 results, and for the US they retrieved 430 results. After the first round of sifting by 2 reviewers 46 potentially eligible websites were identified. Of these 19 websites were later excluded, 13 of which only sold in quantities greater than one or to laboratories/hospitals/workplaces, 5 who incorporated contact with a health professional before the sale, and one which was withdrawn from sale between the search and extraction. We identified 23 molecular virus testing services14–36 and 18 antibody testing services14–16,18,19,21,25,26,28,29,31–34,37–40 meeting the inclusion criteria, sold via 27 websites (25 from the UK14–34,37–40 and 2 from the US35,36). One website40 did not appear in the main search, but was mentioned in many UK news articles, so was included in the cohort. Only two websites using home sampling were identified in the US, the first and second to be approved by the FDA for this use.35,36 Basic characteristics of the websites and tests are given in Supplementary Table 1.

The websites consisted of 13 private health clinics,14–17,20,21,24–26,29,36,38,39 four pharmacies,30,32,34,40 four suppliers of a range of direct to consumer testing online,18,22,31,37 three laboratories,23,33,35 two online sexual health specialists19,27 and one supplier of beauty treatments.28 All 23 molecular virus tests were laboratory-based tests with home sampling. Of the 18 antibody tests, 17 were laboratory-based tests with home self-sampling, and one was a point-of-care test.38 The test manufacturer was identifiable for 9/41 (22%) tests, further details are provided in Supplementary Figure 1.

The mean cost of molecular virus testing was £168 (range £65 to £279) in the UK and $135 (range $119 to $150) in the US. The mean cost of antibody tests was £87 (range £55 to £130) in the UK.

The proportion of websites which met each of the criteria for clear communication (outlined in Table 1) is shown in Figure 1, and examples of reporting are given in Box 1.

Figure 1.
Figure 1.

Proportion of home-sampling COVID-19 tests identified which met/did not meet each of the predefined criteria for clear communication to the consumer.

View this table:
Box 1:

Examples of clear/accurate and unclear/potentially misleading website content

Explaining which test and when to test

All 27 websites stated whether the 41 tests for sale were molecular virus tests or antibody tests, of which 40/41 described the test clearly. Guidance on timing of taking the molecular virus tests and the antibody tests was provided by 15/23 (65%)15,17–21,25–27,29–31,34–36 and 16/18 (89%)15,16,18,19,21,25,26,28,29,31,33,34,37–40 websites, respectively. Recommendations on timing and variation in timing of sampling are detailed in Figure 2, with several contrary to current advice or opinion.4

Figure 2.
Figure 2.

Recommendations given by websites on when to take the molecular virus tests and antibody tests. Test accuracy is dependent on correct timing.

Test accuracy and interpretation

Of the 41 tests for sale, the websites reported a measure of test accuracy (sensitivity, specificity, positive or negative predictive value) for 27 (66%) tests: 16/18 (89%) for antibody tests14,15,18,19,21,25,26,28,29,31,33,34,37–40 and 11/23 (48%) for molecular tests.14,17,20–22,25,26,33–36 An additional 10/41 (24%) tests (two antibody16,32 and eight molecular tests15,16,18,19,23,27,29,32) only reported test performance using unclear terms such as ‘accuracy’ or ‘reliability’, for example “This test has a 99.9% accuracy”19 and “This test offers 99.9% reliability”.29 Tests with unclear performance values may be referring to analytical performance, such as “Our test is sensitive to fewer than 100 copies of the target viral RNA, making it a highly accurate test.”32 For two (5%) molecular tests, no text or values referring to accuracy were reported on the websites.24,31

Sensitivity and specificity were the most commonly reported accuracy measures, provided for 27/41 (66%)14,15,17–22,25,26,28,29,31,33–40 and 22/41 (54%)15,17,19–22,25,26,28,29,31,33–37,39,40 tests, respectively. Sensitivity estimates ranged from 95% to 100% for antibody tests (n = 16)14,15,18,19,21,25,26,28,29,31,33,34,37–40 and 97.5% to 100% for molecular tests (n = 11)14,17,20–22,25,26,33–36; specificity estimates ranged from 97.5% to 100% for antibody tests (n = 13)15,19,21,25,26,28,29,31,33,34,37,39,40 and were reported as 100% for all molecular tests (n = 9).17,20–22,25,26,34–36 Five of the 41 tests (13%; two antibody tests28,31 and three molecular14,20,33 tests) provided an estimate or statements of sensitivity and/or specificity under conditions of perfect use rather than pragmatic use, for example “If there are any coronavirus on your swab it will definitely find it”.33

No websites directly referred to positive predictive values (PPV), but they were indirectly reported for 5/41 (12%) tests.25,25,21,20,40 Two antibody25,40 and three molecular tests25,21,20 made a statement about the lack of false positives (implying a PPV of 100%), for example “if it shows a positive result, it can only be for COVID-19”.25 No cross-reactivity (meaning the test would not identify other viruses, only COVID-19 virus) was referred to by websites for 13/41 (32%) tests (five antibody16,25,33,34,40 and eight molecular tests17,20–22,25,26,28,34). Negative predictive value (NPV) was not referred to by any websites; however, statements implying that the NPV was less than 100% were given for 4/41 (10%) available tests (two antibody25,31 and two molecular tests20,35), for example “The test can sometimes show a negative result even if you are infected SARS-CoV-2, the virus that causes COVID-19”.35

The number of samples used to generate accuracy data were given for 5/41 (12%) tests; two antibody tests31,33 and three molecular tests.22,35,36 Accuracy data were linked to a journal publication for only 1/41 (2%) test.33

Information on interpreting both positive and negative molecular virus test results was presented for 4/23 (17%) websites.20,33–35 Twelve of the 18 (67%) websites selling antibody tests informed potential customers prior to purchase that a positive antibody test may not infer immunity from future infection14,16,18,21,25,28,32–34,37,39,40 (Figure 1).

Where tests could be identified, we checked accuracy claims against data from published papers, pre-prints (based on information obtained from searches from ongoing Cochrane reviews for these tests) and manufacturer’s data in the Instructions for Use (IFU) sheet for each test (Table 2). Four websites reported clinical performance data for the Abbott IgG antibody test: two31,33 quoted the performance figures from the IFU41, for the other two26,29 no exact match with available studies could be made. Of the four molecular tests, no performance data were available for the Randox test23 (including in the IFU42), no direct match of clinical performance results could be made for the website selling the Primerdesign genesig PCR assay22 (where the IFU only reported data from contrived samples43), whereas the data reported by US websites35,36 selling the LabCorp and Rutgers PCR tests, respectively, matched data from the manufacturers’ IFUs.44,45

View this table:
Table 2:

Claims of test accuracy from websites (selected verbatim) and evidence identified from the manufacturers’ Instructions for Use sheet (IFU), published papers and pre-prints.

Claims about Regulatory Approval and Endorsement

Across the 25 UK websites, there were 17 antibody tests for home sampling,14–16,18,19,21,25,26,28,29,31–34,37,39,40 one antibody test for home testing38 and 21 molecular tests for home sampling14–34 for sale. There was no mention of regulatory approval or endorsement for 18/39 (46%) tests, seven antibody tests14,16,26,28,32,39,40 and 11 molecular tests14,16,18,19,22–24,27,29–31 (see Supplementary Table 1).

For home sampling antibody tests, 7/17 (41%) included a statement that the test had a CE mark15,19,21,25,33,34,37 and 7/17 (41%) websites included a clear statement that the test had endorsement from a policy making body such as Public Health England, the NHS or the UK or other European government.15,18,21,29,31,33,34 This is despite the fact that currently no COVID-19 antibody tests have regulatory approval for home sampling or home testing. Claims being made about home sampling tests were based on approved test use by health professionals using venous rather than finger-prick samples:

“All of our home test kits are CE-marked. This is one of the two IgG tests approved by the Government for UK use.”15

One website38 claimed it had regulatory approval for its home testing antibody test:

“Our test has been accepted by Medicines and Healthcare products regulatory Agency (MHRA), which means that it can be applied across the EU including UK. We confirm our product can meet the requirement of In vitro Diagnostic Medical Devices Directive (98/79EC) and standards complying with CE Declaration of Conformity.”38

Five of 21 (24%) UK websites selling molecular virus tests for home sampling included a statement that they had regulatory approval15,21,25,26,33 and 6 (29%) websites17,20,28,32–34 claimed approval from a policy making body for this intended use. The manufacturer or name of the molecular tests for which websites were claiming regulatory approval or endorsement could not be identified. Only for two websites selling molecular tests22,23, the test manufacturer could be identified, neither of which made any claims about regulation or endorsement. One of these tests23 is mentioned by the UK government as part of its COVID-19 testing strategy46 and the other22 was one of the tests which was independently evaluated by PHE.47

Both USA websites selling molecular viral tests35,36 have approval from the FDA for home sampling during the COVID-19 pandemic. These websites included information about the eligibility checks that purchasers would need to undergo either prior to purchase or prior to test processing.

We reviewed the 18 UK websites selling home COVID-19 antibody tests14–16,18,19,21,25,26,28,29,31–34,37–40 on 11th/12th June 2020 after the MHRA had instructed sales of these tests to cease because of the lack of approval for the tests using finger-pick samples.11 We found two websites32,38 that appeared to still be selling finger-prick tests, four14,21,28,31 had switched to providing a venous blood sampling service, two18,33 required the purchaser to find their own phlebotomist to draw a blood sample to send, six15,16,19,25,34,40 simply stated that tests were out of stock and were unavailable, whilst four26,29,37,39 reported the MHRA guidance and indicated that they had suspended sales (Table 3).

View this table:
Table 3:

Availability of finger-prick antibody tests post MHRA withdrawal from market notice (websites accessed on 11th to 12th June 2020)

DISCUSSION

We identified 27 websites selling COVID-19 tests direct to the public, 25 in the UK but only two in the US, which may be explained by the FDA stipulations requiring clinician involvement in the testing process. We observed that many websites failed to provide complete information on the name and manufacturer of the test (no information for 32/41 tests), when to use the test (no information for 10/41 tests), the accuracy of the test (no information for 12/41 tests), and how to interpret results (no information for 21/41 tests), which will hinder the public making informed choices about testing, using tests correctly and understanding what test results mean. Without adequate and correct information the public may purchase the wrong or a poor test, or use the test in the wrong way or at the wrong point in time. These errors or application will increase their chances of getting an erroneous test result. Even when used properly, few websites assisted users in interpreting test results and understanding their inherent uncertainty.

This rapid evaluation was designed to provide timely results in the context of a fast-moving global pandemic. The search was not designed to be exhaustive, rather to represent what a person typing “coronavirus test” or similar into a Google search would have retrieved. Using different phrases such as “coronavirus antibody test” would have identified additional websites, but there is no reason to suspect that they would be different from those summarised here. The timing of the search and data extraction will have affected results. Data extraction was shortly before the UK MHRA clarified that antibody tests were not approved for finger-prick samples, only for venous samples. The search only identified two US websites selling tests with home sample collection, but at the point of going to press eight tests are now approved on the FDA website.48 The criteria that we used to assess completeness of communication (detailed in Table 1) were defined a priori, but due to time constraints a formal process for developing these was not followed. However, all key elements of the search, selection and data extraction processes were undertaken independently by two researchers, reducing the possibility of errors. We only assessed information provided prior to purchase, as complete information should be given at this stage to inform the purchasing decision. However, further information would have been given after purchase, for example within the instructions for use, which was beyond the scope of this paper.

The issues we have identified are examples of poor and misleading practice, and some merit further investigation by the MHRA and Advertising Standards Authority. At the time of going to press two antibody tests remained on sale. The communication of test accuracy appears to contravene advertising standards in the UK. The five websites that reported PPV of 100% contrary to the wider evidence base, and all websites making accuracy claims which is not linked to supporting evidence appear to contravene section 12.1 of the ASA code,49 which states that objective claims must be backed by evidence. Further, websites provided specificity and sensitivity, or general claims of ‘accuracy’ rather than positive and negative predictive values explained in lay terms, and the ASA have previously ruled against this practice as misleading in the case of non-invasive prenatal testing for trisomies.50 Finally, the lack of complete information on the implications of positive and negative test results does not appear to be covered by any UK regulation, perhaps because the ASA 12.2.149 prohibit diagnosis by post or email, and so this information is intended to be provided by contact with a healthcare professional. Whilst such contact with a health professional is happening in the US it does not appear to be in the UK. Regulation of product labelling provides a means to oversee information communicated for self-testing products bought in person, but there is currently no equivalent for online testing services in the UK. This gap in regulation could be solved by expanding the responsibility of the MHRA to include communication by ‘distributors’ at the point of online purchase, working collaboratively with the Advertising Standards Agency. There was a large variation in price of testing in the UK, and in many cases these differences do not appear to be justified by differences in the service provided. Greater regulation and standardisation of website claims may reduce this price differential by making comparisons between websites easier, and removing unsubstantiated claims.

Key Communication Requirements

It is important that all test users are given adequate and appropriate information to help them make safe and informed choices. We identified five key communication issues with websites selling direct to consumer home-sampling COVID-19 tests. All five of these issues may be improved by developing a basic framework of what information should be provided, and standard ways to present such information. This would also facilitate comparison between websites.

1) The type of test and the questions which it can help address

It is essential that companies selling tests identify the type of test, and the situations in which it is appropriate to order such a test. Whilst websites were clear whether they were selling molecular or antibody tests, they also need to indicate the situations when it is appropriate to order a molecular “swab” test or an antibody “blood” test in order to select the correct one. The two US websites utilised questionnaires recording symptoms and exposure which were reviewed by clinicians prior to tests being despatched which provides a more rigorous check on whether the test request is sensible.

2) How and when the test should be used

Both molecular and antibody tests need to be used at different time points in the disease course. The sensitivity of both types of tests will fall if used at the wrong time point (sensitivity of 31% for antibody tests in the first week since onset of symptoms3), substantially increasing the risks of infection or antibody response being undetected. Recommended time points when samples should be taken were absent for 10/39 UK tests (26%). Some timing statements were misleading, suggesting using the test at time points which are known to be too early or too late. Some websites stated dates based on time since exposure, others since symptom onset which is median 5 days after exposure. Both are required to be able to advise both asymptomatic patients and patients with unknown exposure when they should order and use the tests.

Websites must also describe the full testing process and clearly indicate what is required of users to complete testing. For example, two antibody websites currently indicate that purchasers will need to identify individuals qualified to take venous blood samples, which is impractical for most people.

3) The test name, evidence of its accuracy, and evidence of its regulatory approval for the purpose to which it is put

The majority of tests were for sale by third parties, ranging from healthcare providers to beauty treatment specialists. In most cases (32/41; 78%) it was not possible to identify the test being used or the manufacturer. This does not allow the individual to know the product that they are buying, and precludes the opportunity for the user to verify its regulatory status and the claims being made.

Information on test accuracy was absent or uninterpretable for 12/41 (29%) tests. Numerical accuracy claims could only be matched to published evidence for 4/41 (10%) of tests. In these instances, figures most closely matched those from the manufacturers’ Instructions for Use leaflets, which tended to report the highest observed values of sensitivity and specificity, and were based on studies more akin to analytic validity than clinical validity evaluations. Accuracy measures from analytic validity studies should not be assumed to give a good representation of test accuracy when applied in practice to the public. A wide range of terms were used, several of which did not have a clear meaning. It appeared that test accuracy data is not available at all for some tests (Randox)42, or only based on contrived samples (Primerdesign)43 and not on real patients. For molecular COVID-19 tests, no clinical performance data were available that were based on self-sampled swab tests. Withholding the fact that there is no patient based evidence of the accuracy of these tests from the public is unacceptable. It is important that the reported accuracy is based on all reviewed evidence and not selected results, and clearly explains how applicable the evidence is to the public.

Naming tests is essential to be able to check their regulatory status. Seven of 18 (39%) UK websites selling antibody tests inappropriately claimed CE marking, when the CE marking was for a different intended use (venous rather than finger-prick blood samples). Antibody tests are not approved for home use in the US, and none were found in our search. The UK regulator acted after we had reviewed UK websites, clarifying that antibody tests which are approved for the use with venous samples should not be sold for the use with finger-prick samples. However, 2/18 remain available for online sale at the point of going to press (accessed 11th to 12th June 2020). The molecular tests we could identify are approved for home sampling, however, the name and manufacturer was not identifiable for most websites.

4) What test results mean

Research concerning the communication of test accuracy evidence is limited and is largely restricted to self-selected, professional and postgraduate student groups.51 Communication of test accuracy evidence is complex for several reasons. Research has highlighted the importance of communicating the potential consequences of positive and negative test results (use of predictive values) and the importance of contextualising estimates of accuracy with reference to a healthcare setting (for example hospital in patient, hospital outpatient, community).52,53 Presenting test accuracy as frequencies rather than as probabilities improves understanding.

To interpret results, test users need to know how to interpret positive and negative test results (predictive values), not the proportion of cases detected (sensitivity) and non-cases correctly diagnosed (specificity). Positive predictive value was only reported for 5/41 (12%) tests, and in all five they claimed it was 100% which is inconsistent with the broader evidence base. Negative predictive value was not reported at all.

Most websites gave insufficient information regarding interpreting test results. Only 8/23 (35%) websites explained that a negative molecular virus test does not rule out COVID-19, and only 12/18 (67%) explained that a positive antibody test does not necessarily infer immunity from future COVID-19 infection or transmission.

5) Decisions which could be made based on the test results

Misunderstanding of the implications of test results could mean that individuals put themselves or others at risk of infection in the mistaken belief that they do not have COVID-19, or that they are immune to COVID-19. This last category probably has the greatest potential for harm. Clear communication about the meaning of test results as detailed above should be linked to evidence-based guidance about behaviour modification in light of test results. We found widespread evidence of websites failing to provide such evidence-based guidance, and some cases of websites actively suggesting unsafe behaviour.

CONCLUSIONS

At the point of online purchase of home self-sampling COVID-19 tests, users in the UK are provided with incomplete, and in some cases misleading information on test application, accuracy and interpretation. Many websites omit trustworthy guidance on the timing of tests, the interpretation of positive and negative test results, and the implications of results. Best practice guidance for communication about tests to the public should be developed and the role of the regulator in enforcing complete and accurate information should be reviewed. This should be underpinned by robust collaborative qualitative research exploring how members of the public interpret information and measures of accuracy, thus informing how it can be provided in a way that is clear, complete and accessible

Data Availability

All data are reported in the manuscript. No additional data are available.

Author contributions

All authors contributed to the conception of the work and interpretation of the findings. OO and JG performed the Google searches. STP, SB, KF, JG, OO, IMH, and MJP extracted the data. STP, AJS, MJP and CD undertook the analysis and drafted the manuscript. All authors critically revised the manuscript and approved the final version. STP acts as guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding statement

This paper presents independent research supported by the NIHR Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. Dr Taylor-Phillips is supported by an NIHR Career Development Fellowship (CDF-2016-09-018). MS is supported by the National Institute for Health Research (NIHR) Applied Research Collaboration (ARC) West Midlands. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Competing interests disclosed

We have read and understood BMJ policy on declaration of interests and declare the following interests: STP is funded by the NIHR through a career development fellowship; KF is funded by the NIHR through a doctoral research fellowship; MS reports grants from NIHR Applied Research Collaboration WM; AJS, SB, MP, CD and JD report funding and support from NIHR Birmingham Biomedical Research Centre.

License for publication

I, Sian Taylor-Phillips, the Corresponding Author have the right to grant on behalf of all authors and do grant on behalf of all authors, a worldwide licence to the Publishers and its licensees in perpetuity, in all forms, formats and media (whether known now or created in the future), to i) publish, reproduce, distribute, display and store the Contribution, ii) translate the Contribution into other languages, create adaptations, reprints, include within collections and create summaries, extracts and/or, abstracts of the Contribution, iii) create any other derivative work(s) based on the Contribution, iv) to exploit all subsidiary rights in the Contribution, v) the inclusion of electronic links from the Contribution to third party material where-ever it may be located; and, vi) licence any third party to do any or all of the above.

Ethical approval

Ethical approval not required

Data sharing statement

No additional data available

Transparency

The lead authors and manuscript’s guarantor affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

Acknowledgement

References

  1. 1.↵
    World Health Organization. Laboratory testing for coronavirus disease (COVID-19) in suspected human cases: interim guidance, 19 March 2020. Geneva: World Health Organization, 2020.
  2. 2.↵
    Arevalo-Rodriguez I, Buitrago-Garcia D, Simancas-Racines D, et al. FALSE-NEGATIVE RESULTS OF INITIAL RT-PCR ASSAYS FOR COVID-19: A SYSTEMATIC REVIEW. medRxiv 2020:2020.04.16.20066787. doi: 10.1101/2020.04.16.20066787
    OpenUrlAbstract/FREE Full Text
  3. 3.↵
    Deeks JJ, Dinnes J, Takwoingi Y, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database of Systematic Reviews 2020(6)
  4. 4.↵
    National Health Service (NHS). Ask for an antigen test to check if you have coronavirus now [updated 17 June 2020. Available from: https://www.nhs.uk/conditions/coronavirus-covid-19/testing-and-tracing/ask-for-a-test-to-check-if-you-have-coronavirus/ accessed 19/06/2020.
  5. 5.↵
    Covolo L, Rubinelli S, Ceretti E, et al. Internet-Based Direct-to-Consumer Genetic Testing: A Systematic Review. J Med Internet Res 2015;17(12):e279. doi: 10.2196/jmir.4378 [published Online First: 2015/12/19]
    OpenUrlCrossRefPubMed
  6. 6.↵
    De S, Pietila AM, Iso-Touru T, et al. Information Provided to Consumers about Direct-to-Consumer Nutrigenetic Testing. Public Health Genomics 2019;22(5-6):162–73. doi: 10.1159/000503977 [published Online First: 2019/11/30]
    OpenUrlCrossRef
  7. 7.↵
    Singleton A, Erby LH, Foisie KV, et al. Informed choice in direct-to-consumer genetic testing (DTCGT) websites: a content analysis of benefits, risks, and limitations. J Genet Couns 2012;21(3):433–9. doi: 10.1007/s10897-011-9474-6 [published Online First: 2011/12/24]
    OpenUrlCrossRefPubMed
  8. 8.↵
    The European Parliament and the Council of the European Union. REGULATION (EU) 2017/746 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU 2017 [Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32017R0746&from=EN#d1e4364-176-1 accessed 30/05/2020.
  9. 9.↵
    U.S. Food & Drug Administration. Federal Food, Drug and Cosmetic Act (FD&C Act) Chapter V: Drugs and Devices 2018 [Available from: https://www.fda.gov/regulatory-information/federal-food-drug-and-cosmetic-act-fdc-act/fdc-act-chapter-v-drugs-and-devices accessed 30/05/2020.
  10. 10.↵
    Medicines & Healthcare products Regulatory Agency. Guidance for notified bodies on the regulation of IVDs for self-testing 2012. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/377885/Guidance_for_notified_bodies_on_the_regulation_of_IVDs_for_self-testing.pdf (accessed 02/06/2020).
  11. 11.↵
    Medicine & Healthcare products Regulatory Agency. Guidance For industry and manufacturers: COVID-19 tests and testing kits. Updated 29 May 2020 [Available from: https://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/for-industry-and-manufactures-covid-19-tests-and-testing-kits accessed 30/05/2020.
  12. 12.↵
    Medicine & Healthcare products Regulatory Agency. Guidance For patients, the public and professional users: a guide to COVID-19 tests and testing kits. [Available from: https://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/for-patients-the-public-and-professional-users-a-guide-to-covid-19-tests-and-testing-kits accessed 04/06/2020.
  13. 13.↵
    U.S. Food & Drug Administration. Emergency Use Authorizations [updated 06/04/2020. Available from: https://www.fda.gov/medical-devices/emergency-situations-medicaldevices/emergency-use-authorizations#covid19ivd accessed 30/05/2020.
  14. 14.↵
    CityDoc. Coronavirus Home Testing Kits [Available from: https://www.citydoc.org.uk/covid-19-home-testing-kit/ accessed 25/05/2020.
  15. 15.↵
    DoctorCall. Covid-19 Home Testing Kits [Available from: https://www.doctorcall.co.uk/private-gp-services/coronavirus--covid-19--testing accessed 24/05/2020 and 26/05/2020.
  16. 16.↵
    Summerfield Healthcare. Covid-19 Test kits and Covid-19 Test Antibody kits [Available from: https://www.summerfieldhealthcare.co.uk/corona-virus-testing/ accessed 24/05/2020.
  17. 17.↵
    London Doctors Clinic. Services – Private Cornavirus Test [Available from: https://www.londondoctorsclinic.co.uk/services/private-coronavirus-test/ accessed 25/05/2020.
  18. 18.↵
    Blue Horizon Medicals. COVID-19 Home Coronavirus Test Kits [Available from: https://bluehorizonbloodtests.co.uk/collections/covid-19-coronavirus-tests accessed 24/05/2020.
  19. 19.↵
    Better2Know. Covid-19 testing [Available from: https://www.better2know.co.uk/page/covid-19 accessed 26/05/2020.
  20. 20.↵
    Private Harley Street Clinic. COVID-19 PCR Home Sample Kit – Currently infected [Available from: https://privateharleystreetclinic.com/products/covid-19-test accessed 24/05/2020.
  21. 21.↵
    Qured. Covid-19 PCR and Antibody Tests Now Available [Available from: https://qured.com/covid-19-testing/ accessed 24/05/2020 and 25/05/2020.
  22. 22.↵
    Rightangled. COVID-19 Test [Available from: https://rightangled.co/product/covid-19-test accessed 24/05/2020.
  23. 23.↵
    Randox Health. COVID-19 Home Testing Kit [Available from: https://www.randoxhealth.com/product/coronavirus-covid-19-home-testing-kit/ accessed 26/05/2020.
  24. 24.↵
    SameDayDoctor. COVID-19 Abbott Coronavirus testing [Available from: https://samedaydoctor.org/service/covid-19-coronovirus-antibody-testing/ accessed 24/05/2020.
  25. 25.↵
    Zava. Coronavirus antibody and swab testing service [Available from: https://www.zavamed.com/uk/coronavirus-service.html accessed 22/05/2020.
  26. 26.↵
    TheOnlineClinic. Coronavirus (COVID-19) [Available from: https://www.theonlineclinic.co.uk/coronavirus-testing.asp accessed 27/05/2020.
  27. 27.↵
    Family Planning Association (FPA). Covid-19 Coronavirus Home Test Kit [Available from: https://www.fpa.org.uk/product/new-covid-19-coronavirus-home-test-kit accessed 26/05/2020.
  28. 28.↵
    Antibody Solutions. Covid-19 Home IgG Sample Collection Test Kit and Lab Test [Available from: https://antibodysolutions.co.uk/ accessed 27/05/2020.
  29. 29.↵
    MyHealthcare Clinic. Coronavirus (COVID-19) Testing [Available from: https://myhealthcareclinic.com/covid-19-purchase-tests/ accessed 24/05/2020.
  30. 30.↵
    MedExpress Enterprises Ltd. Coronavirus Testing – Home Sample Collection Kit [Available from: https://www.medexpress.co.uk/testing/covid19/coronavirus-covid19-home-testing-kit-pcr accessed 24/05/2020.
  31. 31.↵
    Atruchecks Ltd. Covid-19 tests – Private Covid-19 Clinical Testing Service [Available from: https://www.atruchecks.com/category/covid-19-tests accessed 27/05/2020.
  32. 32.↵
    PillDoctor. Covid-19 [Available from: https://www.pilldoctor.co.uk/product-category/Covid-19 accessed 27/05/2020.
  33. 33.↵
    London Medical Laboratory. Coronavirus Antibody & PCR Tests – In Stock Now [Available from: https://www.privatecoronavirustests.com/ accessed 24/05/2020.
  34. 34.↵
    Webmed Pharmacy Ltd. covid-19 tests [Available from: https://webmedpharmacy.co.uk/buy/covid19testkit accessed 23/05/2020.
  35. 35.↵
    Pixel by LabCorp. COVID-19 Test (At-Home Kit) [Available from: https://www.pixel.labcorp.com/at-home-test-kits/covid-19-test accessed 27/05/2020.
  36. 36.↵
    Hims Inc. COVID-19 Saliva Test [Available from: https://www.forhims.com/covid-test accessed 28/0/2020.
  37. 37.↵
    Medichecks.com Ltd. Coronavirus Antibody Blood Test [Available from: https://medichecks.com/pages/coronavirus-antibody-blood-test-information accessed 23/05/2020.
  38. 38.↵
    YourHealthFirst Clinic. Ten minute coronavirus self-testing kit [Available from: https://www.yourhealthfirst.uk/our-services/covid-19-rapid-test/ accessed 25/05/2020.
  39. 39.↵
    Babylon. COVID-19 Antibody Test [Available from: https://www.babylonhealth.com/coronavirus/covid-19-antibody-test accessed 23/05/2020.
  40. 40.↵
    Superdrug. Coronavirus Antibody Test Service [Available from: https://onlinedoctor.superdrug.com/coronavirus-antibody-test.html accessed 28/05/2020.
  41. 41.↵
    Abbott Laboratories. SARS-CoV-2 IgG ARCHITECT. Instructions for Use (IFU) April 2020. https://www.fda.gov/media/137383/download.
  42. 42.↵
    Randox Laboratories Ltd. Extended Coronavirus Multiplex Array EV4418 User Manual. https://static1.squarespace.com/static/568969badf40f3987cccc1d9/t/5e6912a5d041d92494d4fdc7/1583944358498/EV4418.doc.pdf.
  43. 43.↵
    Primerdesign Ltd. Coronavirus (COVID-19) genesig®Real-Time PCR assay – For Emergency Use Authorization Only – Instructions for Use (IFU) Issue 1.02. 31 March 2020. https://www.fda.gov/media/136309/download.
  44. 44.↵
    U.S. Food & Drug Administration. ACCELERATED EMERGENCY USE AUTHORIZATION (EUA) SUMMARY – SARS-CoV-2 ASSAY (Rutgers Clinical Genomics Laboratory). 2020. https://www.fda.gov/media/137776/download.
  45. 45.↵
    U.S. Food & Drug Administration. ACCELERATED EMERGENCY USE AUTHORIZATION (EUA) SUMMARY – COVID-19 RT-PCR TEST(LABORATORY CORPORATION OF AMERICA). 2020. https://www.fda.gov/media/136151/download.
  46. 46.↵
    Department of Health & Social Care. Coronavirus (COVID-19) – Scaling up our testing programmes. Department of Health and Social Care. Published 04 April 2020. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/878121/coronavirus-covid-19-testing-strategy.pdf (accessed 11/06/2020).
  47. 47.↵
    National Infection Service (NIS) laboratories Public Health England. Rapid assessment of the Novacyt Primerdesign Coronavirus (COVID-19) genesig® Real-Time PCR assay. June 2020. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/889330/Rapid_assessment_Novacyt_Primerdesign_Coronavirus__COVID-19__Genesig.pdf.
  48. 48.↵
    U.S. Food & Drug Administration. In Vitro Diagnostics EUAs: U.S. Food & Drug Administration; 2020 [updated 18/06/2020. Available from: https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas accessed 19/06/2020.
  49. 49.↵
    The Advertising Standards Authority Ltd. (ASA) and The Committee of Advertising Practice Ltd. (CAP). 12 Medicines, medical devices, health-related products and beauty products – CAP Code. 2020 [Available from: https://www.asa.org.uk/type/non_broadcast/code_section/12.html accessed 20/06/2020.
  50. 50.↵
    The Advertising Standards Authority Ltd. (ASA) and The Committee of Advertising Practice Ltd. (CAP). Non-Invasive Prenatal Testing (NIPT) – A look at the ASA’s rulings. CAP News. 16 Jan 2020 [Available from: https://www.asa.org.uk/news/non-invasive-prenatal-testing-nipt-a-look-at-the-asa-s-rulings.html accessed 20/06/2020.
  51. 51.↵
    Whiting PF, Davenport C, Jameson C, et al. How well do health professionals interpret diagnostic information? A systematic review. BMJ Open 2015;5(7):e008155. doi: 10.1136/bmjopen-2015-008155 [published Online First: 2015/07/30]
    OpenUrlAbstract/FREE Full Text
  52. 52.↵
    Whiting P, Leeflang M, de Salis I, et al. Guidance was developed on how to write a plain language summary for diagnostic test accuracy reviews. J Clin Epidemiol 2018;103:112–19. doi: 10.1016/j.jclinepi.2018.07.008 [published Online First: 2018/07/24]
    OpenUrlCrossRefPubMed
  53. 53.↵
    Whiting P, Davenport C. Understanding test accuracy research: a test consequence graphic. Diagn Progn Res 2018;2:2. doi: 10.1186/s41512-017-0023-0 [published Online First: 2018/02/01]
    OpenUrlCrossRef
  54. 54.
    Lauer SA, Grantz KH, Bi Q, et al. The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application. Ann Intern Med 2020;172(9):577–82. doi: 10.7326/M20-0504 [published Online First: 2020/03/10]
    OpenUrlCrossRefPubMed
  55. 55.
    Bryan A, Pepper G, Wener MH, et al. Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in Boise, Idaho. J Clin Microbiol 2020 doi: 10.1128/JCM.00941-20 [published Online First: 2020/05/10]
    OpenUrlAbstract/FREE Full Text
  56. 56.
    Phipps WS, SoRelle JA, Li Q-Z, et al. SARS-CoV-2 Antibody responses do not predict COVID-19 disease severity. medRxiv 2020:2020.05.15.20103580. doi: 10.1101/2020.05.15.20103580
    OpenUrlAbstract/FREE Full Text
  57. 57.
    Public Health England. Evaluation of the Abbott SARS-CoV-2 IgG for the detection of anti-SARS-CoV-2 antibodies. Version of 8 June 2020. London, 2020:1–18.
  58. 58.
    Epitope Diagnostics Inc. EDI™ Novel Coronavirus COVID-19 IgG ELISA Kit (KT-1032). Instructions for Use (IFU) May 2020. http://www.epitopediagnostics.com/covid-19-elisa.
  59. 59.
    Kruttgen A, Cornelissen CG, Dreher M, et al. Comparison of four new commercial serologic assays for determination of SARS-CoV-2 IgG. J Clin Virol 2020;128:104394. doi: 10.1016/j.jcv.2020.104394 [published Online First: 2020/05/18]
    OpenUrlCrossRefPubMed
  60. 60.
    Public Health England. Rapid assessment of the Randox Extended Coronavirus Multiplex Array assay. V01.20 ed. London, June 2020:1–5.
  61. 61.
    van Kasteren PB, van der Veer B, van den Brink S, et al. Comparison of seven commercial RT-PCR diagnostic kits for COVID-19. J Clin Virol 2020;128:104412. doi: 10.1016/j.jcv.2020.104412 [published Online First: 2020/05/18]
    OpenUrlCrossRefPubMed
Back to top
Posted August 19, 2020.
Download PDF
Data/Code
Email
Information given by websites selling home self-sampling COVID-19 tests: An analysis of accuracy and completeness
S Taylor-Phillips, S Berhane, AJ Sitch, K Freeman, MJ Price, C Davenport, J Geppert, IM Harris, O Osokogu, M Skrybant, JJ Deeks
medRxiv 2020.08.18.20177360; doi: https://doi.org/10.1101/2020.08.18.20177360
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area