Point of care lung ultrasound is useful when screening for CoVid-19 in Emergency Department patients
====================================================================================================

* Paul Walsh
* Andrea Hankins
* Heejung Bang

## Abstract

**Background** CoVid-19 can be a life-threatening lung disease or a trivial upper respiratory infection depending on whether the alveoli are involved. Emergency department (ED) screening in symptomatic patients with normal vital signs is frequently limited to oro-nasopharyngeal swabs. We tested the null hypothesis that patients being screened for CoVid-19 in the ED with normal vital signs and without hypoxia would have a point-of-care lung ultrasound (LUS) consistent with CoVid-19 less than 2% of the time.

**Subjects** Subjects were identified from ED ultrasound logs.

**Inclusion criteria** Age 14 years or older with symptoms prompting ED screening for CoVid-19.

**Exclusion criteria** Known congestive heart failure or other chronic lung condition likely to cause excessive B lines on LUS.

**Intervention** Structured blinded ultrasound review and chart review

**Analysis** We used a two-sided exact hypothesis test for binomial random variables. We also measured LUS diagnostic performance using computed tomography as the gold standard.

**Results** We reviewed 77 charts; 49 met inclusion criteria. Vital signs were normal in 30/49 patients; 10 (33%) of these patients had LUS consistent with CoVid-19. We rejected the null hypothesis (p-value < 0.001). The treating physicians’ interpretation of their own point of care lung ultrasounds had a sensitivity of 100% (95% CI 75%, 100%) and specificity of 80% (95% CI 68%, 89%).

**Conclusion** LUS has a meaningful detection rate for CoVid-19 in symptomatic ED patients with normal vital signs. We recommend at least LUS be used in addition to PCR testing when screening symptomatic ED patients for CoVid-19.

**What is known** Auscultation and chest x-ray are insufficient to screen for lung involvement when SARS-CoV-2 infection is suspected. Point of care lung ultrasound is widely available, safer, and less resource intensive than CT imaging.

**What we found** In symptomatic patients presenting to the ED even those with normal vital signs had point of care lung ultrasound evidence of alveolar level involvement 33%of patients. Point of care lung ultrasound was 100% sensitive and 80% specific compared to CT (reference standard) when evaluating patients for Covid-19.

**What this adds** Point of care lung ultrasound or similar imaging should performed when screening symptomatic patients in whom SARS-CoV-2 infection is suspected.

## Introduction

SARS-CoV-2 causes a variety of respiratory symptoms ranging from pharyngitis or rhinitis, through bronchitis, to multifocal peripheral pneumonitis extending to the alveoli. [1–3] Two clinically important characteristics of SARS-CoV-2 infection are that auscultatory findings may be subtle or normal even in the presence of advanced lower airway disease, and chest x-rays are inadequate for screening. [4] In common with other coronaviruses and influenza, SARS-CoV-2 is likely spread by both the droplet and airborne routes. [5–7] When aerosolized the resulting respirable particles less than 10μ in aerodynamic diameter contain viable virus and can reach adult alveoli directly. [8] Smaller aerosols (5μ) reach the alveoli without also being deposited in the bronchi.[8] This can lead to a clinical picture where a patient has serious lower respiratory tract infection with little or no concomitant upper respiratory tract infection. [6] Nasopharyngeal swabs, even if correctly collected, can therefore fail to detect SARS-CoV-2 and provide false reassurance despite ongoing alveolar destruction.

Screening for SARS-CoV-2 therefore frequently, but not always, includes both viral swabs from the oro-nasopharynx and imaging of the lower respiratory tract. This has included chest x-ray, computed tomography (CT) imaging, and sometimes point of care lung ultrasound. CT scan has been shown to be useful for screening and detecting patients with COVID-19 pneumonia, especially in the highly suspicious, a- or pre-symptomatic cases with negative nucleic acid testing. However, CT imaging is slow, exposes the patient to ionizing radiation, and exposes additional staff to SARS-CoV-2.[4, 9]

Point of care lung ultrasound can detect SARS-CoV-2 induced lung disease, is readily available in most emergency departments, (ED) does not expose the patient to ionizing radiation, and does not require the staff, expertise, and time necessary for traditional CT imaging.[10] Nonetheless, point of care lung ultrasound does add 5 to 10 minutes to the duration of patient evaluation, increases the treating physicians’ exposure to SARS-CoV-2, and decreases the number of patients seen hourly by that physician.

This raises the question as to whether lung imaging could be deferred if the patient being screened for SARS-CoV-2 has normal vital signs. Conversely, if the presence of normal vital signs does not preclude ultrasound evidence of lung disease then some current practices of swab only testing must be considered inadequate.

Our null hypothesis was that among symptomatic patients being screened for CoVid-19 in the emergency department that the point of care lung ultrasound would be consistent with CoVid-19 less than 2% of the time if vital signs were normal.

We also measured the diagnostic performance of point of care lung ultrasound compared with chest x-ray and CT chest. For comparative purposes we also measured the diagnostic performance of chest x-ray and crackles or rales on auscultation with CT chest.

## Methods

### Study Design

Cross sectional study with structured chart and ultrasound imaging review.

### Subjects

Patients aged 14 years old and older screened for CoVid-19 in an adult and pediatric emergency department were considered. Subjects were identified from the imaging archive of the ED ultrasound machine. Patients attended from 4-March-2020 to 19-May-2020. Patients had LUS performed if the treating physician was facile in point-of care LUS, believed that lung imaging should form part of CoVid-19 screening, and did not send the patient for immediate computed tomography of the chest. Although not randomly selected, the patients would not systematically differ from patients not imaged.

### Ultrasound imaging protocol

The physicians performing the point of care lung ultrasound typically imaged the posterior acoustic windows by running the US probe down the patient’s back midway between the scapula and vertebral column. Axillary and anterior windows were typically interrogated with single views of each. Physicians sometimes chose to not interrogate all possible windows if they had already reached their diagnosis on the windows already imaged. Images were captured with a Zonare Z One Ultra Ultrasound machine. The probes available for use were linear 10.5 MHz, linear 4.1 MHz, and curvilinear 9.3 MHz

### Inclusion Criteria

Images archived with adequate identifiers, age 14 years or older, and being evaluated for SARS-CoV-2 infection causing a CoVid-19 illness.

### Exclusion Criteria

Patients were excluded for a prior medical history of congestive heart failure or other chronic lung disease likely to affect point of care lung ultrasound interpretation (for example disease likely to cause B lines or pleural thickening) and if the point of care lung ultrasound was performed for a reason other than evaluating for CoVid-19. We did not exclude patients with a history of asthma or chronic obstructive pulmonary disease.

### Study definitions

We defined symptomatic as the documentation of any of the following in medical record: cough, subjective fever, fatigue, weakness, sore throat, or shortness of breath, nausea or vomiting, diarrhea, sore throat, fatigue, or headache.

Abnormal vital signs- Any pulse or respiratory rate at or above the 98th centile for age for children [11] For adults, tachycardia was defined as pulse at or above 100 beats per minute, tachypnea as respiratory rate above 22 breaths per minute, fever as temperature as at or above 38°Centigrade, and hypotension as systolic blood pressure at or below 80 mm Hg.[12] We did not have an upper limit for blood pressure. We included oxygen saturation measured by pulse oximetry as a vital sign and defined hypoxia as oxygen saturation of less than 92%.

We accepted the interpretation of the ultrasound by performing physician as consistent with CoVid-19 or viral pneumonitis for our primary analysis. If the performing physician did not document an interpretation we substituted the blinded reading.

### Data abstraction

One investigator (PW) performed a blinded reading of all LUS images using a structured template prior to performing chart review. Another, (AH) extracted data from EPIC/Clarity electronic health records using via SQL Server Management Studio.

Vital signs for each visit and polymerase chain reaction (PCR) results of swabs were extracted from their respective fields in an EPIC electronic health record. Only the first set of vital signs was retained. Vital signs and lab results were directly extracted from the electronic health record.

The full text of the ED visit was downloaded into a text file. EPIC electronic health record periodically automatically saves even incomplete notes as they are entered. The time of each, even incomplete note, is recorded. This allowed us to ensure the ultrasound note was entered before the CT scan resulted. However, the resulting downloaded files are duplicative and large (text fields alone average 1 megabyte per patient).

The ultrasound note was typically entered either in free-form or using personalized physician-created templates. These were in various locations in the chart. Some were typed into distinct stand-alone progress notes, others were included in the main chart, still others were included in progress notes that included other patient information. We used a simplified sentiment analysis (sentimentR) in R to locate the bedside ultrasound report in the chart.[13] This created an html page highlighting text that sentiment analysis considered to be an ultrasound report. In three cases a bedside ultrasound report could not be found using either this semi-automated technique or a manual chart review and the blinded interpretation was substituted.

CT scan and chest x-ray results have standardized headers and were located using regexm functions in Stata and then manually reviewed and data abstracted using a standardized template by an author (PW).

Because there was only one chart reviewer, inter rater reliability was not a concern. We did not attempt intra-rater reliability measurement of the chart abstraction process.

## Data and statistical analysis

We tested the null hypothesis using the bitest command in Stata. This performs exact hypothesis tests for binomial random variables. The null hypothesis was that the probability of a positive ultrasound was 2%. Our sample size calculations are shown in Appendix 1.

We compared inter rater reliability between the treating physician and reader relying on only the archived images using Gwet’s agreement coefficient (AC1). Gwet’s AC1’s validity does not depend upon the hypothesis of independence between raters and it does not result in unexpectedly low values (as seen in Cohen’s κ) when agreement is expected to be high.[14,15] We have previously shown how Cohen’s κ can be misleading in pediatric emergency medicine research and why alternatives such as Gwet’s AC1 should often be used instead.[16] We used kappaetc in Stata to calculate Gwet’ AC1. [17] We measured diagnostic performance of the point of care lung ultrasound using board certified radiologists’ interpretations of the CT chest as the gold standard using the diagt command in Stata. [18]

Data and statistical analysis was performed using Stata 16.1 (Statacorp LLP, College Station TX and R (R Foundation for Statistical Computing, Vienna, Austria).

## Ethical approval

The institutional review and privacy boards for Sutter Health approved this study and granted a waiver of informed consent (approval number 1597263).

## Results

We identified 77 point of care lung ultrasound scans with usable data of which 49 met our inclusion and exclusion criteria. All 77 scans were used to measure inter rater reliability and diagnostic performance characteristics. All the point of care lung ultrasounds were performed before the CT scans. Fig 1 shows patient flow through the study. The demographic characteristics of subjects are shown in Table 1.

View this table:
[Table 1.](http://medrxiv.org/content/early/2020/06/12/2020.06.09.20123836/T1)

Table 1. 
Clinical characteristics of study patients. IQR, interquartile range; LUS lung ultrasound.

![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/06/12/2020.06.09.20123836/F1.medium.gif)

[Figure 1.](http://medrxiv.org/content/early/2020/06/12/2020.06.09.20123836/F1)

Figure 1. 
Patient flow through the study.

The treating physician interpreted 18/49 (37%) point of care lung ultrasounds as being consistent with Covid-19. Vital signs were normal in 30 patients and 10 (33%) of these patients had LUS consistent with CoVid-19.

We therefore reject the null hypothesis that among symptomatic patients being screened for CoVid-19 in the emergency department that the point of care lung ultrasound would be consistent with CoVid-19 less than 2% of the time if vital signs were normal (p-value < 0.001). We accept our alternative hypothesis that point of care lung ultrasound would be consistent with CoVid-19 more than 2% of the time even if the vital signs are normal.

When compared with the subsequent CT, the treating physicians’ interpretation of their own point of care lung ultrasounds had a sensitivity of 100% (95% CI 75%, 100%) and specificity of 80% (95% CI 68%, 89%). For the over reading physician relying only on archived images the sensitivity and specificity were 92% (95% CI 62%, 100%) and 37% (95% CI 25%, 50%) respectively. The performance characteristics of point of care lung ultrasound using CT chest as the gold standard are detailed in Table 2.

View this table:
[Table 2.](http://medrxiv.org/content/early/2020/06/12/2020.06.09.20123836/T2)

Table 2. 
Comparison of diagnostic performance of bedside point of care lung ultrasound, chest x ray, and crackles on auscultation for diagnosis of lung involvement of SARS-CoV-2 using CT chest as the gold standard. Predictive values were based on 16% prevalence. Sens, sensitivity; CI, confidence interval, Spec, specificity; PPV, positive predictive value; NPV, negative predictive value; LR +, likelihood ratio positive; LR –, likelihood ratio negative; AUC, area under the receiver operator curve.

Inter rater agreement measured using Gwet’s AC1 between the bedside physician who performed the point of care lung ultrasound and the over-reading physician using only archives was 68%. Most characteristics showed acceptable inter-rater reliability between the bedside read and images over read. Excess short non-coalescent B-lines and pleural thickening showed poor agreement likely reflecting both the subjectivity of these items and the difference between reviewing saved and real-time images.

View this table:
[Table 3.](http://medrxiv.org/content/early/2020/06/12/2020.06.09.20123836/T3)

Table 3. 
Inter rater agreement between a blinded over read relying only on saved images and the bedside interpretation of the treating physician. CI, confidence interval. Where the readings differed the bedside physician ultrasonographer interpretation was used.

PCR testing for SARS-CoV-2 was not always available and when it was a variety of tests performed at different sites were used. The results are shown in Table 4.

View this table:
[Table 4.](http://medrxiv.org/content/early/2020/06/12/2020.06.09.20123836/T4)

Table 4. 
PCR results from nasal, nasopharyngeal, and or oropharyngeal swabs. Although the overall number of PCR tests was the same, some patients received SARS-CoV-2 testing alone, others had a panel of respiratory pathogens ordered without SARS-CoV-2 due to lack of test availability at the time. PCR, polymerase chain reaction; the panel of respiratory pathogens tested included Adenovirus, Parainfluenza viruses 1-4, Mycoplasma pneumoniae, Bordetella pertussis, Coronaviruses 229E, HKU1, N163 and OC43, RSV, respiratory syncytial virus; HMV, human metapneumovirus; Chlamydiaphilia, Chlamydiaphilia pneumoniae; US, ultrasound; HMV, human metapneumovirus.

## Discussion

Point of care lung ultrasound detected lesions consistent with alveolar involvement in 33% of symptomatic patients with normal vital signs who were being screened for CoVid-19.

These findings are consistent with published case series and social media reports of the utility of lung ultrasound in the diagnosis of CoVid-19.[19, 20] The use of point of care lung ultrasound in Covid-19 screening has been spontaneous and sporadic practice tending typically occurring in emergency medicine and critical care. Some radiologists have also found lung ultrasound useful.[19, 20] Regardless of the specialty, point of care lung ultrasound practices in the detection of CoVid-19 have necessarily evolved ahead of their published evidence base. The peer reviewed literature is sparse. Previous literature has comprised of case reports, and case series of 12 and 20 patients.[20, 21],[22] Scanning techniques, and images of patients with proven Covid-19 have spread among clinicians on Twitter and blogs[23, 24] and many others.

Our results may not be viewed so favorably to those who argue for implementation of CT scanning as the initial screening test. A CT only imaging approach by definition, will lead to better detection rates among those images than LUS. Such an approach, however, is expensive and slow and places a greater number of hospital staff at risk of infection, although it results in less infected patient exposure time for the doctor. It is possible that dedicated portable chest CT scanners using three slice protocols to save time and radiation would be a better alternative. This approach would require resources that are much less widely available than ED ultrasound.

Lung ultrasound has emerged as a clinical tool in human and veterinary medicine, and animal research with some advocates calling for it to replace the stethoscope. [25–28] Others have shown ultrasound to complement rather than replace the physical exam and to correlate reasonably well with lung findings at necropsy. Ultrasound decreases CT utilization in inpatients with suspected CoVid-19.[29] Descriptive papers have found ultrasound correlates well with CT and clinical characteristics in CoVid-19 patients. [30, 31] Recommendations for training novices to identify CoVid-19 have been published.[29] Ultrasound cannot be expected to replace CT imaging, but the ease with which it can be performed serially and at the bedside, and its apparent sensitivity makes it a useful tool for screening for alveolar level disease in SARS-CoV-2 infection.

Knowing whether a patient has alveolar involvement with CoVid-19 is clinically important. Patients’ initially mild lung disease has been shown to progress, sometimes rapidly, on serial CT scans as the disease progresses. [32] Patients frequently are unaware of their own deterioration and may present, or fail to re-present with critically low oxygen saturations without overt symptoms. These patients frequently have negative PCR tests unless bronchoalveolar lavage is performed. Such patients risk being falsely reassured about their own impending fate, and continue to infect others when, inevitably, they cough.

## Limitations

This was a single center study and was not a random sample. Whether a patient was seen by a physician who both believed that CoVid-19 screening should include lung imaging and was facile with ultrasound was a matter of luck, but not randomization. This adds uncertainty to estimates of the prevalence of pneumonitis that point of care lung ultrasound can detect among patients being screened for CoVid-19. Other limitations of our work include its small sample size, and a single chart reviewer. Patients with mild disease, and especially those with normal vital signs did not always have CT imaging performed. PCR testing for SARS-CoV-2 was not always available and even when it was the ideal gold standard of bronchoalveolar lavage was not performed.

Our use of CT scan as a gold standard is imperfect as CT diagnosis of CoVid-19 has its own limitations.[33] It is difficult to conceive of an alternative gold standard that does not fall foul of circular reasoning (by for example using ‘two out of three’ imaging methods positive as the gold standard). Moreover, given the false negative rates of PCR testing, CT rather than PCR testing has been recommended as the primary diagnostic modality in high prevalence settings. [34]

Our assessment of the performance characteristics of ultrasound is limited by our sample size. The performance characteristics of lung ultrasound were not the primary concern of this manuscript however. Although falling out of favor, null hypothesis testing is well suited to answering our primary question when the sample size is small– after all, single brown (red) Holstein demolishes the hypothesis that all cows are black and white, and careful planning minimizes the number of cows that need to be seen.

Despite these limitations, we can be assured that the prevalence of pneumonitis in these patients is more than the 2% ‘acceptable miss rate’ for high morbidity conditions and this may be sufficient to adjust practice accordingly. [35] Other limitations include the use of abbreviated LUS imaging protocols and the variability in image saving practices with doctors saving many cine-clips, and others only one or two still images. These differences in practice style could decrease inter rater agreement between the blinded and bedside readings. Much more detailed and formalized lung ultrasound protocols and ultrasound scoring systems specifically for use in SARS-CoV-2 patients have been described.[31, 36] Abbreviated protocols are inevitable in community practice and could lead to missed diagnoses. This would bias our study in the opposite direction of our actual findings.

## Conclusion

In conclusion, point of care lung ultrasound has a meaningful detection rate for alveolar level involvement in SARS-CoV-2 infection when screening symptomatic emergency department patients with normal vital signs. We recommend at least point of care lung ultrasound be used in addition to PCR testing when screening symptomatic patients in whom SARS-CoV-2 infection is suspected.

## Data Availability

Statistical code used in the analysis is available but the data is not but a public use dataset may be released with a peer-reviewed publication.

## Acknowledgements

The authors acknowledge Kimberly Kolbias, DO, a visiting resident from Desert Regional Medical Center, who posed the question “Do I really need to do the ultrasound if the vital signs are normal?” “Yes, you do.” This project was supported in part by the Pediatric Emergency Medicine Research Foundation, (Walsh) and by The National Institutes of Health through grant UL1 TR001860 (Bang). The ultrasound machine was provided by The Sutter Medical Foundation.

*   Received June 9, 2020.
*   Revision received June 9, 2020.
*   Accepted June 12, 2020.


*   © 2020, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)

## References

1.  1.Chan JF, Yuan S, Kok KH, To KK, Chu H, Yang J, et al. A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster. Lancet. 2020;395(10223):514–23. Epub 2020/01/24. doi: 10.1016/S0140-6736(20)30154-9. PubMed PMID: 31986261; PubMed Central PMCID: PMCPMC7159286.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(20)30154-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31986261&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

2.  2.Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395(10223):497–506. Epub 2020/01/24. doi: 10.1016/S0140-6736(20)30183-5. PubMed PMID: 31986264; PubMed Central PMCID: PMCPMC7159299.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(20)30183-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31986264&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

3.  3.Wang Z, Yang B, Li Q, Wen L, Zhang R. Clinical Features of 69 Cases with Coronavirus Disease 2019 in Wuhan, China. Clin Infect Dis. 2020. Epub 2020/03/16. doi: 10.1093/cid/ciaa272. PubMed PMID: 32176772.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciaa272&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32176772&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

4.  4.Yoon SH, Lee KH, Kim JY, Lee YK, Ko H, Kim KH, et al. Chest Radiographic and CT Findings of the 2019 Novel Coronavirus Disease (COVID-19): Analysis of Nine Patients Treated in Korea. Korean J Radiol. 2020;21(4):494–500. Epub 2019/02/26. doi: 10.3348/kjr.2020.0132. PubMed PMID: 32100485; PubMed Central PMCID: PMCPMC7082662.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3348/kjr.2020.0132&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32100485&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

5.  5.Booth TF, Kournikakis B, Bastien N, Ho J, Kobasa D, Stadnyk L, et al. Detection of airborne severe acute respiratory syndrome (SARS) coronavirus and environmental contamination in SARS outbreak units. J Infect Dis. 2005;191(9):1472–7. Epub 2005/03/18. doi: 10.1086/429634. PubMed PMID: 15809906.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/429634&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15809906&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000228128800014&link_type=ISI) 

6.  6.Tellier R, Li Y, Cowling BJ, Tang JW. Recognition of aerosol transmission of infectious agents: a commentary. BMC Infect Dis. 2019;19(1):101. Epub 2019/01/31. doi: 10.1186/s12879-019-3707-y. PubMed PMID: 30704406; PubMed Central PMCID: PMCPMC6357359.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12879-019-3707-y&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

7.  7.Yu IT, Li Y, Wong TW, Tam W, Chan AT, Lee JH, et al. Evidence of airborne transmission of the severe acute respiratory syndrome virus. N Engl J Med. 2004;350(17):1731–9. doi: 10.1056/NEJMoa032867. PubMed PMID: 15102999.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa032867&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15102999&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000220947300007&link_type=ISI) 

8.  8.Institute of Medicine United States. Preventing Transmission of Pandemic Influenza and Other Viral Respiratory Diseases: Personal Protective Equipment for Healthcare Personnel: Update 2010.
    
    

9.  9.Meng H, Xiong R, He R, Lin W, Hao B, Zhang L, et al. CT imaging and clinical course of asymptomatic cases with COVID-19 pneumonia at admission in Wuhan, China. J Infect. 2020. Epub 2020/04/12. doi: 10.1016/j.jinf.2020.04.004. PubMed PMID: 32294504; PubMed Central PMCID: PMCPMC7152865.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jinf.2020.04.004&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32294504&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

10. 10.Lung ultrasound findings in patients with novel SARS-CoV2 [Internet]. MedArchiv. 2020.
    
    

11. 11.Siberry G, Iannone R, Childs B. Harriet Lane Handbook: A manual for pediatric house officers. 15 ed.  America dobmdbtlDSo, editor. St Louis: Mosby; 2000.
    
    

12. 12.Tobin MJ, Chadha TS, Jenouri G, Birch SJ, Gazeroglu HB, Sackner MA. Breathing patterns. 1. Normal subjects. Chest. 1983;84(2):202–5. doi: 10.1378/chest.84.2.202. PubMed PMID: 6872603.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1378/chest.84.2.202&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=6872603&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1983RB41500022&link_type=ISI) 

13. 13.Rinker T. sentimentr. 2018. Available at [https://mran.microsoft.com/snapshot/2018-03-30/web/packages/sentimentr/sentimentr.pdf](https://mran.microsoft.com/snapshot/2018-03-30/web/packages/sentimentr/sentimentr.pdf) Accessed 8-June-2020
    
    

14. 14.Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology. 2008;61(1):29–48. doi: 10.1348/000711006X126600.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1348/000711006X126600&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18482474&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

15. 15.Gwet KL. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters: Advanced Analytics Press; 2012.
    
    

16. 16.Walsh P, Thornton J, Asato J, Walker N, McCoy G, Baal J, et al. Approaches to describing inter-rater reliability of the overall clinical appearance of febrile infants and toddlers in the emergency department. PeerJ. 2014;2:e651. Epub 2014/11/11. doi: 10.7717/peerj.651. PubMed PMID: 25401054; PubMed Central PMCID: PMCPMC4230550.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7717/peerj.651&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25401054&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

17. 17.Klein D. Implementing a general framework for assessing interrater agreement in Stata. The Stata Journal: Sage; 2018. p. 871–901.
    
    

18. 18.Seed P. DIAGT: Stata module to report summary statistics for diagnostic tests compared to true disease status. Boston College Department of Economics; 2001.
    
    

19. 19.Buonsenso D, Piano A, Raffaelli F, Bonadia N, de Gaetano Donati K, Franceschi F. Point-of-Care Lung Ultrasound findings in novel Coronavirus disease-19 pnemoniae: a case report and potential applications during COVID-19 outbreak. Eur Rev Med Pharmacol Sci. 2020;24(5):2776–80. doi: 10.26355/eurrev\_202003\_20549. PubMed PMID: 32196627.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access\_num=10.26355/eurrev_202003_20549&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32196627&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

20. 20.Poggiali E, Dacrema A, Bastoni D, Tinelli V, Demichele E, Mateo Ramos P, et al. Can Lung US Help Critical Care Clinicians in the Early Diagnosis of Novel Coronavirus (COVID-19) Pneumonia? Radiology. 2020:200847. Epub 2020/03/13. doi: 10.1148/radiol.2020200847. PubMed PMID: 32167853.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200847&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32167853&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

21. 21.A Preliminary Study on the Ultrasonic Manifestations of Peripulmonary Lesions of Non-Critical Novel Coronavirus Pneumonia (COVID-19) [Internet]. [https://www.ssrn.com/index.cfm/en/](https://www.ssrn.com/index.cfm/en/). 2020.
    
    

22. 22.Thomas A, Haljan G, Mitra A. Lung ultrasound findings in a 64-year-old woman with COVID-19. CMAJ. 2020;192(15):E399. Epub 2020/03/31. doi: 10.1503/cmaj.200414. PubMed PMID: 32234724; PubMed Central PMCID: PMCPMC7162441.
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTkyLzE1L0UzOTkiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMC8wNi8xMi8yMDIwLjA2LjA5LjIwMTIzODM2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

23. 23.Chen YT. Physician patient with Covid. In: @yaletung, editor. Day 5 after COVID diagnosis Less cough & tired, still no dyspnea/chest pain #POCUS update: Effusion resolved, as subpleural consolidations spread bilaterally on both posterior lower lobes Started on HCQ yesterday: [Twitter.com](http://Twitter.com); 2020.
    
    

24. 24.Rodriguez P. Day 1 Covid_19 symptoms with my ButterflyNetlnc SatO2 98%, fever, dry cough and tiredness. POCUS: thick pleura & B-Lines in lower lobes. In: @pablitorf, editor. US clips Serial exams of a physician patient with CoVid-19: Twitter; 2020.
    
    

25. 25.Buczinski S, Forte G, Francoz D, Belanger AM. Comparison of thoracic auscultation, clinical score, and ultrasonography as indicators of bovine respiratory disease in preweaned dairy calves. J Vet Intern Med. 2014;28(1):234–42. Epub 2013/11/19. doi: 10.1111/jvim.12251. PubMed PMID: 24236441.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/jvim.12251&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24236441&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

26. 26.Copetti R. Is lung ultrasound the stethoscope of the new millennium? Definitely yes! Acta Med Acad. 2016;45(1):80–1. doi: 10.5644/ama2006-124.162. PubMed PMID: 27284804.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.5644/ama2006-124.162&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27284804&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

27. 27.Grune J, Beyhoff N, Hegemann N, Lauryn JH, Kuebler WM. From bedside to bench: lung ultrasound for the assessment of pulmonary edema in animal models. Cell Tissue Res. 2020;380(2):379–92. Epub 2020/02/03. doi: 10.1007/s00441-020-03172-2. PubMed PMID: 32009189; PubMed Central PMCID: PMCPMC7210222.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00441-020-03172-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32009189&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

28. 28.Lichtenstein DA, Malbrain MLNG. Lung ultrasound in the critically ill (LUCI): A translational discipline. Anaesthesiol Intensive Ther. 2017;49(5):430–6. Epub 2017/11/18. doi: 10.5603/AIT.a2017.0063. PubMed PMID: 29151003.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.5603/AIT.a2017.0063&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29151003&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

29. 29.Vetrugno L, Bove T, Orso D, Barbariol F, Bassi F, Boero E, et al. Our Italian experience using lung ultrasound for identification, grading and serial follow-up of severity of lung involvement for management of patients with COVID-19. Echocardiography. 2020;37(4):625–7. Epub 2020/04/15. doi: 10.1111/echo.14664. PubMed PMID: 32239532.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/echo.14664&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

30. 30.Smith MJ, Hayward SA, Innes SM, Miller ASC. Point-of-care lung ultrasound in patients with COVID-19 – a narrative review. Anaesthesia. 2020. Epub 2020/04/10. doi: 10.1111/anae.15082. PubMed PMID: 32275766.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/anae.15082&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32275766&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

31. 31.Soldati G, Smargiassi A, Inchingolo R, Buonsenso D, Perrone T, Briganti DF, et al. Is There a Role for Lung Ultrasound During the COVID-19 Pandemic? J Ultrasound Med. 2020. Epub 2020/03/20. doi: 10.1002/jum.15284. PubMed PMID: 32198775.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/jum.15284&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32198775&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

32. 32.Xu X, Yu C, Qu J, Zhang L, Jiang S, Huang D, et al. Imaging and clinical features of patients with 2019 novel coronavirus SARS-CoV-2. Eur J Nucl Med Mol Imaging. 2020;47(5):1275–80. Epub 2020/02/28. doi: 10.1007/s00259-020-04735-9. PubMed PMID: 32107577; PubMed Central PMCID: PMCPMC7080117.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00259-020-04735-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

33. 33.Bai HX, Hsieh B, Xiong Z, Halsey K, Choi JW, Tran TML, et al. Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology. 2020:200823. Epub 2020/03/10. doi: 10.1148/radiol.2020200823. PubMed PMID: 32155105.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200823&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32155105&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

34. 34.Ai T, Yang Z, Hou H, Zhan C, Chen C, Lv W, et al. Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology. 2020:200642. Epub 2020/02/26. doi: 10.1148/radiol.2020200642. PubMed PMID: 32101510.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200642&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32101510&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

35. 35.Kline JA, Johnson CL, Pollack CV, Diercks DB, Hollander JE, Newgard CD, et al. Pretest probability assessment derived from attribute matching. BMC Med Inform Decis Mak. 2005;5:26. Epub 2005/08/11. doi: 10.1186/1472-6947-5-26. PubMed PMID: 16095534; PubMed Central PMCID: PMCPMC1201143.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1472-6947-5-26&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16095534&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F06%2F12%2F2020.06.09.20123836.atom) 

36. 36.Benchoufi M, Bokobza J, Anthony Chauvin A, Dion E, Baranne M-L, Levan F, et al. Lung injury in patients with or suspected COVID-19 : a comparison between lung ultrasound and chest CT-scanner severity assessments, an observational study. medRxiv. 2020:2020.04.24.20069633. doi: 10.1101/2020.04.24.20069633.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNC4yNC4yMDA2OTYzM3YzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDYvMTIvMjAyMC4wNi4wOS4yMDEyMzgzNi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)