A Large-Scale Clinical Validation Study Using nCapp Cloud Plus Terminal by Frontline Doctors for the Rapid Diagnosis of COVID-19 and COVID-19 pneumonia in China ================================================================================================================================================================ * Dawei Yang * Tao Xu * Xun Wang * Deng Chen * Ziqiang Zhang * Lichuan Zhang * Jie Liu * Kui Xiao * Li Bai * Yong Zhang * Lin Zhao * Lin Tong * Chaomin Wu * Yaoli Wang * Chunling Dong * Maosong Ye * Yu Xu * Zhenju Song * Hong Chen * Jing Li * Jiwei Wang * Fei Tan * Hai Yu * Jian Zhou * Jinming Yu * Chunhua Du * Hongqing Zhao * Yu Shang * Linian Huang * Jianping Zhao * Yang Jin * Charles A. Powell * Yuanlin Song * Chunxue Bai ## Abstract **Background** The outbreak of coronavirus disease 2019 (COVID-19) has become a global pandemic acute infectious disease, especially with the features of possible asymptomatic carriers and high contagiousness. It causes acute respiratory distress syndrome and results in a high mortality rate if pneumonia is involved. Currently, it is difficult to quickly identify asymptomatic cases or COVID-19 patients with pneumonia due to limited access to reverse transcription-polymerase chain reaction (RT-PCR) nucleic acid tests and CT scans, which facilitates the spread of the disease at the community level, and contributes to the overwhelming of medical resources in intensive care units. **Goal** This study aimed to develop a scientific and rigorous clinical diagnostic tool for the rapid prediction of COVID-19 cases based on a COVID-19 clinical case database in China, and to assist global frontline doctors to efficiently and precisely diagnose asymptomatic COVID-19 patients and cases who had a false-negative RT-PCR test result. **Methods** With online consent, and the approval of the ethics committee of Zhongshan Hospital Fudan Unversity (approval number B2020-032R) to ensure that patient privacy is protected, clinical information has been uploaded in real-time through the New Coronavirus Intelligent Auto-diagnostic Assistant Application of cloud plus terminal (nCapp) by doctors from different cities (Wuhan, Shanghai, Harbin, Dalian, Wuxi, Qingdao, Rizhao, and Bengbu) during the COVID-19 outbreak in China. By quality control and data anonymization on the platform, a total of 3,249 cases from COVID-19 high-risk groups were collected. These patients had SARS-CoV-2 RT-PCR test results and chest CT scans, both of which were used as the gold standard for the diagnosis of COVID-19 and COVID-19 pneumonia. In particular, the dataset included 137 indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, 62 suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results, and 122 asymptomatic cases who had positive RT-PCR test results, amongst whom 31 cases were diagnosed. We also integrated the function of a survey in nCapp to collect user feedback from frontline doctors. **Findings** We applied the statistical method of a multi-factor regression model to the training dataset (1,624 cases) and developed a prediction model for COVID-19 with 9 clinical indicators that are fast and accessible: ‘Residing or visiting history in epidemic regions’, ‘Exposure history to COVID-19 patient’, ‘Dry cough’, ‘Fatigue’, ‘Breathlessness’, ‘No body temperature decrease after antibiotic treatment’, ‘Fingertip blood oxygen saturation ≤93%’, ‘Lymphopenia’, and ‘C-reactive protein (CRP) increased’. The area under the receiver operating characteristic (ROC) curve (AUC) for the model was 0.88 (95% CI: 0.86, 0.89) in the training dataset and 0.84 (95% CI: 0.82, 0.86) in the validation dataset (1,625 cases). To ensure the sensitivity of the model, we used a cutoff value of 0.09. The sensitivity and specificity of the model were 98.0% (95% CI: 96.9%, 99.1%) and 17.3% (95% CI: 15.0%, 19.6%), respectively, in the training dataset, and 96.5% (95% CI: 95.1%, 98.0%) and 18.8% (95% CI: 16.4%, 21.2%), respectively, in the validation dataset. In the subset of the 137 indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, the model predicted 132 cases, accounting for 96.4% (95% CI: 91.7%, 98.8%) of the cases. In the subset of the 62 suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results, the model predicted 59 cases, accounting for 95.2% (95% CI: 86.5%, 99.0%) of the cases. Considering the specificity of the model, we used a cutoff value of 0.32. The sensitivity and specificity of the model were 83.5% (95% CI: 80.5%, 86.4%) and 83.2% (95% CI: 80.9%, 85.5%), respectively, in the training dataset, and 79.6% (95% CI: 76.4%, 82.8%) and 81.3% (95% CI: 78.9%, 83.7%), respectively, in the validation dataset, which is very close to the published AI model. The results of the online survey ‘Questionnaire Star’ showed that 90.9% of nCapp users in WeChat mini programs were ‘satisfied’ or ‘very satisfied’ with the tool. The WeChat mini program received a significantly higher satisfaction rate than other platforms, especially for ‘availability and sharing convenience of the App’ and ‘fast speed of log-in and data entry’. **Discussion** With the assistance of nCapp, a mobile-based diagnostic tool developed from a large database that we collected from COVID-19 high-risk groups in China, frontline doctors can rapidly identify asymptomatic patients and avoid misdiagnoses of cases with false-negative RT-PCR results. These patients require timely isolation or close medical supervision. By applying the model, medical resources can be allocated more reasonably, and missed diagnoses can be reduced. In addition, further education and interaction among medical professionals can improve the diagnostic efficiency for COVID-19, thus avoiding the transmission of the disease from asymptomatic patients at the community level. ## Introduction The first outbreak of coronavirus disease 2019 (COVID-19) was recorded at the end of 2019 in Wuhan city, Hubei Province, China, and currently more than 185 countries and territories have reported COVID-19 cases.1 According to the latest statistics, more than 4.02 million cases have been diagnosed worldwide, with the majority of cases now in the U.S.2 To prevent and control COVID-19, and to halt the spread of the disease as quickly as possible, it is crucial to provide early detection, timely diagnosis, and proper management of COVID-19 cases. However, the clinical manifestations of COVID-19 vary at different disease stages, and the sensitivity of viral nucleic acid diagnostic tests cannot ensure 100% identification of infected patients. Therefore, chest radiology was added to the diagnostic standard in the fifth edition of the *Chinese COVID-19 diagnosis and treatment guideline* (‘Chinese Guideline’).3 In the space of just two months, the Chinese Guideline has been updated to the seventh edition4 to provide better guidance for doctors in the diagnosis and treatment of COVID-19. The wide spread of COVID-19 and large number of newly infected patients require doctors from different specialties (including general practitioners [GPs]) to participate in the diagnosis, treatment, and management of the disease. The earliest guidelines from the World Health Organization (WHO)5 and China4 listed multiple clinical manifestations and laboratory tests. In combination with expert opinion, there were 20 diagnosis-related factors in total, which made it difficult for non-expert doctors to decide which ones were the most important determinants. As a result, there was a long learning curve for frontline doctors to acquire the skills for the precise and timely diagnosis of the disease. The complexity of diagnosis led to missed diagnoses or delayed diagnoses, and to some extent, facilitated the spread of COVID-19 at the community level.6 Additionally, due to the lack of a modern and standardized medical quality control system, there were missed diagnoses or misdiagnoses, especially among patients with false-negative nucleic acid test results or false-negative CT scan results.7 Currently, there is no rapid clinical diagnosis model specially designed for COVID-19 based on clinical informatics. To create such a model, it is necessary to optimize and screen the essential diagnostic determinants from the existing diagnosis-related factors, and to assist clinical practice by applying user-friendly internet tools. To address this gap, we developed the New Coronavirus Intelligent Auto-diagnostic Assistant Application of cloud plus terminal (nCapp), a mobile-based tool as a mini program within WeChat (a social media App), exploiting the fact that almost all doctors in China are smartphone users. We conducted a prospective clinical study (ClinicalTrials.gov registration number [NCT04275947](http://medrxiv.org/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT04275947&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F11%2F2020.08.07.20163402.atom)). We aimed to apply nCapp, with the assistance of modern statistical methods, to optimize and select a group of essential diagnostic determinants for COVID-19, with high sensitivity and specificity, and to support doctors in their clinical practice, facilitating better management of COVID-19 patients. ## Methods ### Basics of nCapp This study applied nCapp, a mobile-based diagnostic tool, to conduct a multi-center clinical study.8 In addition to following the standards for designing a mobile App, the back-end was developed using Java, and Object Storage Service (OSS) was used for data storage (Figure 1). nCapp applies statistical methods to optimize and select the essential diagnostic determinants for COVID-19. It also provides an easy and accessible intelligent process system to assist doctors from all specialties in their clinical practice. To better describe its essence and function, we use the name ‘nCapp Cloud Plus Terminal’.9 It is the first global mobile application developed for the early diagnosis of infectious diseases, especially COVID-19. To be specific, cases are classified into different risk stratification categories, in line with the *Expert consensus on COVID-19 pneumonia diagnosis and treatment using Intelligent Auto-Diagnostic Assistant Application (nCappf* and differentiated interventions are tiered to each category. For example, interventions for critical cases include a cloud link to senior doctors, symptomatic treatment, respiratory support, circulation support, psychological guidance, and traditional Chinese medicine (TCM) treatment; interventions for suspected cases include reporting or transfer to a superior hospital, a cloud link to senior doctors, and continuation of the current treatment; interventions for indeterminate cases include a cloud link to senior doctors, continuation of the current treatment, and isolation and observation for 14 days. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F1) Figure 1: Flow chart of data information processing through the nCapp Cloud Plus Terminal model ### Data collection The nCapp Cloud Plus Terminal model can be used for many diseases and public health events. It has both Chinese and English versions, and it can be accessed by both iPhone and Android users. The users, who are frontline doctors participating in this study, first need to search for ‘WeChat’ and install the WeChat App on their phones (iPhone users install through the ‘AppStore’; Android phone users install through the ‘PlayStore’). The users then follow the instructions in WeChat to register an account and use the ‘scan’ function to scan the QR code in Figure S1 to start nCapp. Patient privacy is protected with the required online consent and the study protocol was approved by the ethics committee of Zhongshan Hospital Fudan University (approval number B2020-032R). Since the start of the study on February 13, 2020, clinical information on 3,249 cases from COVID-19 high-risk groups has been uploaded through nCapp, with their SARS-CoV-2 reverse transcription-polymerase chain reaction (RT-PCR) nucleic acid test results and chest CT scan results available. The training dataset of the statistical model randomly included 1,624 cases, and the remaining 1,625 cases were included in the validation dataset. The patients were of COVID-19 high-risk groups from Wuhan, Shanghai, Harbin, Dalian, Wuxi, Qingdao, Rizhao, and Bengbu, who were admitted to local infectious disease hospitals. All data were recorded anonymously and uploaded by local doctors. ![Figure S1](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F9.medium.gif) [Figure S1](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F9) Figure S1 QR code for nCapp ### Statistical methods The data on 3,249 COVID-19 patients uploaded through nCapp were input into the dataset. We applied stratified random sampling to sample 1,624 patients for the training dataset of the statistical model, and the remaining 1,625 cases were included in the validation dataset (i.e. all the samples were first divided into two strata based on the RT-PCR results; 606 cases with positive RT-PCR results and 1,018 cases with negative RT-PCR results were randomly included in the training dataset, and the remaining 1,625 cases were included in the validation dataset). SAS 9.4 and R 3.6.2 statistical software packages were used for data analysis and plotting. Odds ratios (ORs) and 95% confidence intervals (CIs) (as the approximate estimate of the relative risk) were reported to estimate the strength of association between each variable and the test result. Single-factor logistic regression analysis was first applied to the training dataset. The RT-PCR test result was entered into the model as a dependent variable (Y: negative=0, positive=1). The effects of different diagnostic factors were ranked based on the results from a single factor analysis, with 0.05 as the significance level for factor inclusion and 0.1 as the significance level for factor exclusion. Independent variables were selected using the step-forward method of partial maximum likelihood estimation. Then stepwise multi-factor logistic regression analysis was applied to obtain the probability model for cases with positive test results (i.e. COVID-19 patients). The goodness of fit of the model on the training dataset and the validation dataset was used to measure the prediction accuracy. ## Results ### Demographics Table 1 demonstrates that the basic demographic characteristics of the cases included in this study approximately coincide with the clinical characteristics of COVID-19 patients in China (Guan et al.), which certifies the reliability of the samples and the generalizability of this research. View this table: [Table 1:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/T1) Table 1: Demographics of the training dataset and the validation dataset Table 2 presents the comparison of demographic indicators among four subsets: cases with positive RT-PCR and CT scan results, cases with a positive RT-PCR result and negative CT scan result, cases with a negative RT-PCR result and positive CT scan result, and cases with negative RT-PCR and CT scan results. It shows statistically significant differences in all the demographic indicators (*P*<0.001), except for sex. View this table: [Table 2:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/T2) Table 2: Demographics of the four subsets ### Case composition and factors affecting the four subsets #### Composition of all the cases As illustrated in Figure 2, among the asymptomatic patients (no dry cough, fever, breathlessness, or fatigue), cases with negative RT-PCR and CT scan results account for the vast majority (79.7%), and the other three subsets account for a low proportion. Among the symptomatic patients (at least one symptom among dry cough, fever, breathlessness, and fatigue), cases with positive RT-PCR and CT scan results account for the highest proportion (41.7%), followed by cases with negative RT-PCR result and positive CT scan result, and cases with negative RT-PCR and CT scan results (both over 20%), while cases with positive RT-PCR result and negative CT scan result account for the lowest proportion (5%). Combined with Figure 3, it shows that ‘no symptoms’ and ‘no exposure history to COVID-19 patient’ are sensitive assessment indicators for cases with negative RT-PCR and CT scan results; ‘with symptoms’ and ‘residing or visiting history in epidemic regions’ or ‘visiting history to Hubei or exposure history to patients with respiratory symptoms’ indicate a higher risk of cases with positive RT-PCR and CT scan results. It is also noteworthy that 13.4% of the asymptomatic cases have positive RT-PCR test results while 53.3% of the symptomatic cases have negative RT-PCR test results. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F2) Figure 2: Composition of the cases (%) ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F3) Figure 3: Clustering of the factors affecting the four subsets ### Probability model for COVID-19 Independent variables in the model of diagnostic factors were selected using the step-forward method of partial maximum likelihood estimation, with 0.05 as the significance level for factor inclusion, and 0.1 as the significance level for factor exclusion. After the single factor logistic regression analysis, a multi-factor regression analysis was applied, including ‘Residing or visiting history in epidemic regions’, ‘Exposure history to COVID-19 patient’, ‘Visiting history to Hubei or exposure history to patient with respiratory symptoms’, ‘Dry cough’, ‘Fever’, ‘No body temperature decrease after antibiotic treatment’, ‘Fatigue’, ‘Breathlessness’, ‘Fingertip blood oxygen saturation ≤93%’, ‘Lymphopenia’, and ‘CRP increased’. We obtained a probability model for cases with positive test results (i.e. COVID-19 patients). The equation expression for the model is: Probability of COVID-19 = ex / (1 + ex) x = −3.13 + (0.99 * ‘Residing or visiting history in epidemic regions’) + (1.15 * ‘Exposure history to COVID-19 patient’) + (0.37 * ‘Dry cough’) + (0.52 * ‘Fatigue’) + (0.74 * ‘Breathlessness’) + (0.53 * ‘Fingertip blood oxygen saturation ≤93%’) + (0.58 * ‘Lymphopenia’) + (0.36 * ‘No body temperature decreased after antibiotic treatment’) + (1.10 * ‘CRP increased’). (Table 3) View this table: [Table 3:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/T3) Table 3: Probability model for COVID-19 based on logistic regression analysis The AUC for the model was 0.88 (95% CI: 0.86, 0.89) in the training dataset (1,624 cases) and 0.84 (95% CI: 0.82, 0.86) in the validation dataset (1,625 cases). To ensure the sensitivity of the model, we used a cutoff value at 0.09. The sensitivity and specificity of the model were 98.0% (95% CI: 96.9%, 99.1%) and 17.3% (95% CI: 15.0%, 19.6%), respectively, in the training dataset, and 96.5% (95% CI: 95.1%, 98.0%) and 18.8% (95% CI: 16.4%, 21.2%), respectively, in the validation dataset. The model predicted 132 cases of the 137 indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, i.e. 96.4% (95% CI: 91.7%, 98.8%) in this subset, and 59 cases of the 62 suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results, i.e. 95.2% (95% CI: 86.5%, 99.0%) in this subset. Considering the specificity of the model, we used a cutoff value at 0.32. The sensitivity and specificity of the model were 83.5% (95% CI: 80.5%, 86.4%) and 83.2% (95% CI: 80.9%, 85.5%), respectively, in the training dataset, and 79.6% (95% CI: 76.4%, 82.8%) and 81.3% (95% CI: 78.9%, 83.7%), respectively, in the validation dataset, which is very close to the published AI model (sensitivity:84.3%, specificity:82.8%).10 ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F4.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F4) Figure 4: Goodness of fit of the probability model for COVID-19 in the training dataset and the validation dataset ### Comparison between the probability model and the clinical diagnosis Figure 5 illustrates that the sensitivity of clinical diagnosis was 12.4% (17/137, 95% CI: 6.9%-17.9%) in the subset of 137 indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, and 17.7% (11/62, 95% CI: 8.2%-27.3%) in the subset of 62 suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results. In comparison, the sensitivity of the probability model for COVID-19 in these two subsets were 96.4% (132/137, 95% CI: 91.7%-98.8%) and 95.2% (59/62, 95% CI: 86.5%-99.0%), respectively. The probability model demonstrated better performance in sensitivity than the clinical diagnosis, which is of statistical significance (*P*<0.05). ![Figure 5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F5.medium.gif) [Figure 5:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F5) Figure 5: Comparison between the probability model and the clinical diagnosis ### Diagnostic value of the model in different subsets As indicated in Figure 6, the diagnosis percentage of the model for asymptomatic patients was 83.6% (51/61, 95% CI: 74.3%-92.9%) in the validation dataset, 100% (16/16, 95% CI: 79.4%-100%) in the subset of indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results and 71.4% (5/7, 95% CI: 29.0%-96.3%) in the subset of suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results. Figure 7 shows that the diagnosis percentage of the model was 96.4% (132/137, 95% CI: 91.7%-98.8%) in the COVID-19 patients who initially did not have RT-PCR tests and subsequently had positive RT-PCR results; to break this down further: 96.2% (125/130, 95% CI: 91.2%-99.7%) in patients with pneumonia, and 100% (7/7, 95% CI: 76.8%-100%) in patients without pneumonia; the diagnosis percentage was 95.2% (59/62, 95% CI: 86.5%-99.0%) in the subset of COVID-19 patients who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results; to break this down further: 100% (53/53, 95% CI: 96.4%-100%) in patients with pneumonia and 66.7% (6/9, 95% CI: 29.9%-92.5%) in patients without pneumonia. The probability model is of high value in the diagnosis of different subsets of COVID-19 cases. ![Figure 6:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F6.medium.gif) [Figure 6:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F6) Figure 6: Diagnostic value of the model in asymptomatic COVID-19 patients ![Figure 7:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F7.medium.gif) [Figure 7:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F7) Figure 7: Diagnostic value of the model in COVID-19 patients with or without pneumonia ### User feedback on nCapp We applied the ‘Questionnaire Star’ online survey for nCapp Cloud Plus Terminal to collect feedback from the terminal users, and 380 questionnaires were collected (40.6% of the total nCapp registered users). The survey covered the basic information of the user, nCapp working environment, and satisfaction. In addition, users were required to rank different platforms based on their own experience, including WeChat mini programs, mobile Apps, desktop Apps, and statistics websites or webpages. The results showed that significantly more users were ‘satisfied’ or ‘very satisfied’ with the WeChat mini program. In summary, the internet-based WeChat mini program nCapp received highly positive feedback from Chinese doctors (overall satisfaction rate, 90.9%). The ‘availability and sharing convenience of the App’ and ‘fast speed of log-in and data entry’ were the most valued features of nCapp, which had significantly better performance compared with other telemedicine tools. (Figure 8) ![Figure 8:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/11/2020.08.07.20163402/F8.medium.gif) [Figure 8:](http://medrxiv.org/content/early/2020/08/11/2020.08.07.20163402/F8) Figure 8: Evaluation of user experience ## Discussion This study has demonstrated that the user-friendly mobile-based tool nCapp can quickly and easily synchronize and share clinical diagnosis and treatment information for research, that is, it helps with the optimization and screening of the determinants required for the diagnosis of COVID-19 while doctors are engaged in clinical practice.8 By applying classical statistical methods to optimize and select the diagnostic determinants, nCapp facilitates the identification and diagnosis of COVID-19 patients and prompts advice on better management of the cases, especially suspected cases who had initial false-negative RT-PCR test results, or indeterminate cases who initially did not have RT-PCR tests as they did not meet the criteria for suspected cases. These features of nCapp can ultimately contribute to the early detection and timely quarantine of patients. Survey results from users indicated that nCapp had significantly better performance in ‘availability and sharing convenience of the App’ and ‘fast speed of log-in and data entry’ compared with other telemedicine tools. Currently more than 4.02 million COVID-19 cases have been reported worldwide, with a continuing trend of transmission at the community level6, which implies a certain number of patients with hidden infections in the epidemic areas11. The pathogenesis of COVID-19 is different from that of SARS or other viral diseases, and evidence shows that a patient’s viral load can last longer12, while cases of recurrence and secondary infection in patients who have been ‘cured’ of COVID-19 have also been reported13. Therefore, it is essential to identify suspected and indeterminate cases with false-negative RT-PCR results, to avoid missed diagnoses or misdiagnoses due to false-negative results, and to provide early quarantine of the infectious sources to halt the spread of the disease. It is also noteworthy that COVID-19 patients with initial false-negative RT-PCR test results are a high risk for the general population14. Our study has demonstrated that nCapp can reduce missed diagnoses, including in the 137 indeterminate cases and 62 suspected cases in our dataset. Additionally, COVID-19 presents challenges for health systems. There is an urgent need to train doctors (including GPs) to have a good understanding of the clinical manifestations, laboratory test results, and diagnostic criteria of COVID-19, and to enhance standardized protocols for diagnosis and treatment15. To address these challenges, it is necessary to simplify the current complex diagnosis-related factors and develop convenient and user-friendly intelligent assistant tools, which enable all doctors to quickly learn and apply for the timely identification, diagnosis, and management of suspicious patients. However, the current consensus guidelines include multiple clinical manifestations and laboratory tests for the diagnosis of COVID-19. To avoid missed diagnoses and to facilitate accessibility in areas with poor medical conditions, a total of 20 factors are regarded as relevant diagnostic factors4. The large number of diagnostic factors becomes a barrier for doctors from different specialties to make comprehensive judgments. For less experienced doctors, it is difficult to differentiate disease-related factors and important diagnostic determinants. Therefore, it is necessary to optimize and select the key determinants that are indispensable to the diagnosis for doctors from different specialties (including GPs) in the clinical practice, thus reducing missed diagnoses. We applied the statistical method of a multi-factor linear regression model and found that the multiple diagnostic factors can be optimally reduced to 9 items, which will be more convenient for doctors to apply in clinical practice, especially with the support of nCapp. This raised another critical question of whether new problems with missed diagnoses and misdiagnoses will emerge after the optimal reduction of diagnosis-related factors to 9 items. We addressed this concern by carefully screening the model, and results showed that the diagnosis percentage of the model for asymptomatic patients was 83.6% (95% CI: 74.3%, 92.9%) in the validation set, 100% (95% CI: 79.4%, 100%) in the subset of indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, and 71.4% (95% CI: 29.0%, 96.3%) in the subset of suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results. As indicated in Figure 7, the diagnosis percentage of the model was 96.4% (95% CI: 91.7%, 98.8%) in the subset of COVID-19 patients who initially did not have RT-PCR tests and subsequently had positive RT-PCR results; to break this down further: 96.2% (95% CI: 91.2%, 99.7%) in patients with pneumonia, and 100% (95% CI: 76.8%, 100%) in patients without pneumonia; the diagnosis percentage was 95.2% (95% CI: 86.5%, 99.0%) in the subset of COVID-19 patients who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results; to break this down further: 100% (95% CI: 96.4%, 100%) in patients with pneumonia, and 66.7% (95% CI: 29.9%, 92.5%) in patients without pneumonia. Characterized by universality and versatility, nCapp can quickly and easily synchronize and share clinical diagnosis and treatment information for research, providing an accessible and convenient platform for real-world research16. The nCapp WeChat mini program is a new open-source cloud platform specially designed for clinical researchers. A mini program can be developed in a short time and easily accessed and distributed among WeChat users17. Different from traditional applications, a WeChat mini program does not need any configuration, which is of great convenience. Clinical researchers and doctors can log in to the program within a few seconds, complete the registration and electronic consent forms at the same time, and immediately start to collect first-hand information and conduct online big-data analyses by applying statistical models. This system facilitates standardized healthcare management, and users can easily implement the clinical guidelines in real time18 following the standardized diagnosis and treatment required by the Chinese Guideline. nCapp has advantages in its functions compared with other COVID-19-related Apps that solely collect patients’ activity information and geographic location through the smartphone’s GPS signal19. Survey results from ‘Questionnaire Star’ representing the feedback of users showed that nCapp integrated a standardized case reporting system with an intelligent assistant diagnosis and treatment system, which enabled it to have significantly better performance in ‘availability and sharing convenience of the App’ and ‘fast speed of log-in and data entry’ than other telemedicine tools, and to contribute to addressing the issues in the current clinical diagnosis of COVID-19. In particular, in some low- and middle-income countries, it is possible to make up for the lack of diagnostic reagents at an early stage of the outbreak by using nCapp, which features convenience and accessibility, and to reduce the risk of widespread people-to-people transmission due to the inability to identify suspicious patients timely, which is of great significance for the global control of the COVID-19 pandemic.16 nCapp terminal users include frontline doctors from different specialties, the staff of the Chinese Centers of Disease Control, and medical professionals at Public Health Centers in China. Through the application of the program, more confirmed, suspected, and indeterminate cases can be tested, quarantined, and managed timely to provide early alert and isolation of diagnosed or suspicious sources of infection. One limitation of this study is that the uploaded cases were mainly moderate-to-severe COVID-19 cases, which leads to restrictions on the application of nCapp for mild patients. In addition, WeChat is not widely used in European or American countries, despite its popularity as a social media cloud platform in China and its emerging application for medical purposes. To address this issue, one approach would be to popularize WeChat, or to develop tools on a similar platform such as WhatsApp for the diagnosis and treatment of COVID-19, in compliance with HIPAA regulations to protect patient data privacy. ## Data Availability N/A ## Footnotes * **Funding:** Shanghai Municipal Key Clinical Specialty(shslczdzk02201) and Shanghai Top-Priority Clinical Key Disciplines Construction Project (2017ZZ02013) * Received August 7, 2020. * Revision received August 7, 2020. * Accepted August 11, 2020. * © 2020, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.WHO. Novel Coronavirus (2019-nCoV) situation reports. 2020 [cited 2020 Feb 9th]. Available from: [https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports](https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports). 2. 2.Coronavirus COVID-19 Global Cases by Johns Hopkins CSSE. Available from: [https://gisanddata.maps.arcgis.eom/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6](https://gisanddata.maps.arcgis.eom/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6). 3. 3.China NHCoPsRo. Guideline of Treatment on new coronavirus infectious pneumonia (5 th edition); 2020. 4. 4.China NHCoPsRo. Guideline of Treatment on new coronavirus infectious pneumonia (7th edition); 2020. 5. 5.WHO. Clinical management of severe acute respiratory infection when novel coronavirus (nCoV) infection is suspected; 2020. 6. 6.Moghadas SM, Shoukat A, Fitzpatrick MC, Wells CR, Sah P, Pandey A, Sachs JD, Wang Z, Meyers LA, Singer BH, Galvani AP. Projecting hospital utilization during the COVID-19 outbreaks in the United States. Proc Natl Acad Sci U S A 2020. 7. 7.Guan W-j, Ni Z-y, Hu Y, Liang W-h, Ou C-q, He J-x, Liu L, Shan H, Lei C-l, Hui DSC, Du B, Li L-j, Zeng G, Yuen K-Y, Chen R-c, Tang C-l, Wang T, Chen P-y, Xiang J, Li S-y, Wang J-l, Liang Z-j, Peng Y-x, Wei L, Liu Y, Hu Y-h, Peng P, Wang J-m, Liu J-y, Chen Z, Li G, Zheng Z-j, Qiu S-q, Luo J, Ye C-j, Zhu S-y, Zhong N-s. Clinical Characteristics of Coronavirus Disease 2019 in China. New England Journal of Medicine 2020. 8. 8.Bai L, Yang D, Wang X, Tong L, Zhu X, Zhong N, Bai C, Powell CA, Chen R, Zhou J, Song Y, Zhou X, Zhu H, Han B, Li Q, Shi G, Li S, Wang C, Qiu Z, Zhang Y, Xu Y, Liu J, Zhang D, Wu C, Li J, Yu J, Wang J, Dong C, Wang Y, Wang Q, Zhang L, Zhang M, Ma X, Zhao L, Yu W, Xu T, Jin Y, Wang X, Wang Y, Jiang Y, Chen H, Xiao K, Zhang X, Song Z, Zhang Z, Wu X, Sun J, Shen Y, Ye M, Tu C, Jiang J, Yu H, Tan F. Chinese experts’ consensus on the Internet of Things-aided diagnosis and treatment of coronavirus disease 2019 (COVID-19). Clinical eHealth 2020; 3: 7–15. 9. 9.Song Y, Jiang J, Wang X, Yang D, Bai C. Prospect and application of Internet of Things technology for prevention of SARIs. Clinical eHealth 2020; 3: 1–4. 10. 10.Mei X, Lee H-C, Diao K, Huang M, Lin B, Liu C, Xie Z, Ma Y, Robson PM, Chung M, Bernheim A, Mani V, Calcagno C, Li K, Li S, Shan H, Lv J, Zhao T, Xia J, Long Q, Steinberger S, Jacobi A, Deyer T, Luksza M, Liu F, Little BP, Fayad ZA, Yang Y. Artificial intelligence-enabled rapid diagnosis of COVID-19 patients. medRxiv 2020: 2020.2004.2012.20062661 11. 11.Li Q, Guan X, Wu P, Wang X, Zhou L, Tong Y, Ren R, Leung KSM, Lau EHY, Wong JY, Xing X, Xiang N, Wu Y, Li C, Chen Q, Li D, Liu T, Zhao J, Liu M, Tu W, Chen C, Jin L, Yang R, Wang Q, Zhou S, Wang R, Liu H, Luo Y, Liu Y, Shao G, Li H, Tao Z, Yang Y, Deng Z, Liu B, Ma Z, Zhang Y, Shi G, Lam TTY, Wu JT, Gao GF, Cowling BJ, Yang B, Leung GM, Feng Z. Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-Infected Pneumonia. New England Journal of Medicine 2020 12. 12.Kim JY, Ko J-H, Kim Y, Kim Y-J, Kim J-M, Chung Y-S, Kim HM, Han M-G, Kim SY, Chin BS. Viral Load Kinetics of SARS-CoV-2 Infection in First Two Patients in Korea. J Korean Med Sci 2020; 35. 13. 13.Chen D, Xu W, Lei Z, Huang Z, Liu J, Gao Z, Peng L. Recurrence of positive SARS-CoV-2 RNA in COVID-19: A case report. Int J Infect Dis 2020; 93: 297–299. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijid.2020.03.003&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32147538&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F11%2F2020.08.07.20163402.atom) 14. 14.Ai T, Yang Z, Hou H, Zhan C, Chen C, Lv W, Tao Q, Sun Z, Xia L. Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology; : 200642. 15. 15.Razai MS, Doerholt K, Ladhani S, Oakeshott P. Coronavirus disease 2019 (covid-19): a guide for UK GPs. BMJj 2020; 368: m800. 16. 16.Niederman MS, Richeldi L, Chotirmall SH, Bai C. Rising to the Challenge of the Novel SARS-coronavirus-2 (SARS-CoV-2): Advice for Pulmonary and Critical Care and an Agenda for Research. American Journal of Respiratory and Critical Care Medicine; : null. 17. 17.Li WHC, Ho KY, Lam KKW, Wang MP, Cheung DYT, Ho LLK, Xia W, Lam TH. A study protocol for a randomised controlled trial evaluating the use of information communication technology (WhatsApp/WeChat) to deliver brief motivational interviewing (i-BMI) in promoting smoking cessation among smokers with chronic diseases. BMC Public Health 2019; 19: 1083. 18. 18.Liu J, Zhang J, Cheng QJ, Xu JF, Jie ZJ, Jiao Y, Huang Y, Qu JM. [Current status of diagnosis and treatment of community-acquired pneumonia in Shanghai revealed by a questionnaire analysis]. Zhonghua Jie He He Hu Xi Za Zhi 2018; 41: 288–295. 19. 19.Wu JT, Leung K, Leung GM. Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in Wuhan, China: a modelling study. The Lancet 2020; 395: 689–697.