Performance of RAPID noncontrast CT stroke platform in large vessel occlusion and intracranial hemorrhage detection =================================================================================================================== * Vivek Yedavalli * Jeremy J Heit * Seena Dehkharghani * Hafez Haerian * John M. Mcmenamy * Justin Honce * Vincent M Timpone * Christopher Harnain * Andrew Kesselman * Anthony Filly * Adam Beardsley * Brian Sakamoto * Christopher Song * James L Montuori * Benjamin Navot * Francisco Villar Mena * Dan-Victor Giurgiutiu * Felipe Kitamura * Fabricio Oliveira Lima * Henrique Coelho Silva * Francisco José Arruda Mont’Alverne * Gregory W Albers ## Abstract **Background** Noncontrast CT (NCCT) is used to evaluate for intracerebral hemorrhage (ICH) and ischemia in acute ischemic stroke (AIS). Large vessel occlusions (LVOs) are a major cause of AIS, but challenging to detect on NCCT. **Aims** The purpose of this study is to evaluate an AI software called RAPID NCCT Stroke (RAPID, iSchemaView, Menlo Park, CA) for ICH and LVO detection compared to expert readers. **Methods** In this IRB approved retrospective, multicenter study, stand-alone performance of the software was assessed based on the consensus of 3 neuroradiologists and sensitivity and specificity were determined. The platform’s performance was then compared to interpretation by readers comprised of eight general radiologists (GR) and three neuroradiologists (NR) in detecting ICH and hyperdense vessel sign (HVS) indicating LVO. Receiver operating characteristics (ROC) curve was used to evaluate the performance of each reader. P < 0.05 was considered significant. **Results** A total of 244 cases were included. Of the 244, 115 were LVOs and 26 were ICHs. One hundred three cases did not have LVO nor ICH. Stand-alone performance of the software demonstrated sensitivities and specificities of 96.2% and 99.5% for ICH and 63.5% and 95.1% for LVO detection. Compared to all 11 readers and eight GR readers only respectively, the software demonstrated superiority, achieving significantly higher sensitivities (63.5% versus 43.6%, p < 0.0001 and 63.5% versus 40.9%, p =0.001). **Conclusion** The RAPID NCCT Stroke platform demonstrates superior performance to radiologists for detecting LVO from a NCCT. Use of this software platform could lead to earlier LVO detection and expedited transfer of these patients to a thrombectomy capable center. Key words * Noncontrast CT * LVO detection * Acute Ischemic Stroke * Artificial Intelligence * Stroke ## Introduction Acute stroke secondary to ischemia or hemorrhage is among the leading causes of death and disability worldwide1. Acute ischemic stroke (AIS) comprises the majority of acute strokes (85-90%) 2. Despite accounting for a small fraction of acute stroke, intracerebral hemorrhage (ICH) confers a high risk of mortality2. Neuroimaging is crucial to contemporary stroke management paradigms. Noncontrast CT (NCCT) is the most commonly used imaging modality to screen for intracranial hemorrhage (ICH) and assess for early ischemic changes3. Large vessel occlusions (LVOs) may also be detectable on NCCT through the identification presence of a hyperdense vessel sign (HVS), a variably present feature of intracranial LVO4. NCCT has a high sensitivity for detecting early ICH2 but has a sensitivity of only approximately 40% for detecting early signs of ischemic stroke5. Sensitivity is also roughly 50% for detecting HVS as a marker of LVO4. In order to facilitate management in the hyperacute setting, artificial intelligence (AI) methods are now increasingly used with NCCT to detect both ICH and/or AIS6 and may serve as valuable adjuncts for radiologic evaluation. Many comprehensive stroke centers perform both a NCCT and CTA in patients with a suspected stroke. However, community hospitals frequently obtain only an NCCT scan initially and then discuss the case with a neurologist and/or radiologist then subsequently obtain a CTA in selected cases. This approach can lead to significant delays in LVO diagnosis and lengthen the time to transfer the patient to a thrombectomy center. AI software has the potential to immediately evaluate a NCCT and notify the treating physicians that an LVO is suspected, which could lead to substantial improvements in workflow. ## Aims and Hypotheses RAPID NCCT Stroke (iSchemaView, Menlo Park, CA; RAPID) is a multi-module fully automated AI platform developed to detect both ICH and LVO (defined as occlusion of the middle cerebral artery (MCA) and/or distal internal carotid artery (ICA)) by integrating multiple findings available from the NCCT. We assessed the sensitivity and specificity of this software for detection of ICH and LVO as a) a stand-alone platform, and b) as compared to general radiologists and neuroradiologists. We hypothesize that the AI software outperforms general radiologists (GR) and is non-inferior to neuroradiologists (NR) in a multicenter retrospective evaluation of acutely presenting patients with suspected stroke. ## Methods ### Subjects This retrospective, multicenter study was conducted in compliance with the Health Insurance Portability and Accountability Act (HIPAA) and was approved by the institutional review boards (IRB; Advarra central IRB Pro00049230). Cases were obtained from consecutive emergency room (ER) scans obtained in patients suspected of having an acute stroke in two of the centers and from research studies that enrolled acute LVO and ICH patients. In total, 244 cases were included in the analysis. ### Definitions of pathology ICH was defined as any type of intracranial hemorrhage including intraparenchymal, subdural, epidural, subarachnoid, and intraventricular hemorrhages5. LVO was defined as distal intracranial ICA and M1 segment of the MCA7. Lastly, HVS was determined to be a segmental hyperdensity which is comparatively more hyperdense with respect to the contralateral hemisphere and corresponding to the expected location of the distal intracranial ICA and/or M1 segment of the MCA4. ### Scanners This study is composed of cases from different CT vendors. In total, 93 cases were obtained from Siemens scanners (Siemens Healthineers, Erlangen, Germany), 50 from GE Healthcare (GE Healthcare, Wauwatosa, WI, USA) 44 from Phillps (Koninklijke Philips, Amsterdam, Netherlands), 57 from Toshiba (Toshiba, Minato City, Japan). ### RAPID NCCT Stroke Development NCCT imaging data were anonymized and translated into a spatial 3D model.This AI software uses neural networks and automated segmentation techniques based on predefined thresholds for identification of ICH, HVS, and Alberta Stroke Program Early CT Score (ASPECTS). A proprietary algorithm was used to determine if an LVO is likely to be present based on a combination of features derived from the assessment of HVS as well as the specific regions of involvement on ASPECTS modules. The specific modules that are used in RAPID NCCT are RAPID ICH 3.0, RAPID ASPECTS 3.0 and RAPID HVS. Eleven total board-certified GR and NR readers assessed all 244 NCCT scans using local installations of DICOM images through a viewing platform (Osirix, Geneva, Switzerland). Expert readers interpreted the images using only soft tissue kernels with 5 mm slice thickness, including multiplanar reformats. All readers performed interpretations blinded to the AI software results and clinical information. Readers then assessed for ICH, and if ICH was not present, they then assessed for suspected LVO. The reference truth for ICH was based on a consensus of two of three neuroradiologists evaluating the NCCT scan using the same parameters. Scans that expert readers identified with ICH were classified as No LVO for LVO performance assessment. Stand-alone performance was based on the reference LVO assessment which was determined by a consensus of two of three neuroradiologists based on a CTA performed concurrently with the NCCT. CTA assessments were performed on maximum intensity projection (MIP) images based on 3 mm slice thickness using a soft tissue kernel six weeks after the initial NCCT assessment. A board-certified NR (JJH, 10 years of experience) independently reviewed cases deemed positive for LVO to screen for suspected vessel calcification in the distal intracranial ICA or M1 segment of the MCA. This review was based on binary determination of presence or absence of vessel calcification. Cases interpreted as positive for ICH were not assessed for vessel calcifications and classified as ‘No LVO.’ The primary endpoint was sensitivity and specificity of the software as compared to those of the GR and NR readers who were blinded to the CTA results. In total, eight GR and three NR expert readers (different individuals than the experts who determined the reference standard on the CTAs) participated in this phase of the study. Readers were instructed to consider both the presence of HVS as well as early parenchymal signs of brain ischemia when making their determination of LVO. The primary hypothesis was that the automated software would have a higher sensitivity than GR for detecting LVO and be non-inferior to NR readers. If non-inferiority was achieved, then the software would be tested for superiority against all readers as well as NR readers alone. Overall accuracy was also compared between the software and individual readers to assess for both sensitivity and specificity. ### Statistical analysis Sensitivity and specificity analyses were calculated by comparison of the software results to the CTA reference standard for LVO for the stand-alone evaluation. Subsequently, the sensitivity and specificity of the software was compared to the GR and NR readers. Using the ratings (1 to 5) scale that readers used to express their confidence in their LVO call, the readers’ ROC curves were then compared to the overall ROC curves. ## Results Ultimately, a total of 244 cases were included in this study. Of the 244, 115 were LVO (115/244, 47.1%) and 26 (26/244, 10.7%) were ICHs based on the consensus of two of three expert neuroradiologists. One hundred three cases (103/244, 42.2%) were independently reviewed for suspected vessel calcification after initial screening. One hundred three cases (103/244, 42.2%) did not have LVO nor ICH. ### NCCT Stroke stand-alone performance LVO: The software identified 73 true suspected LVOs (73/115, 63.5%) with 42 false negatives (42/115, 36.5%) It also correctly categorized 90 cases (90/115, 78.2%) where LVO was not present with five false positives (5/115, 4.3%). This resulted in a sensitivity of 63.5% (95% CI:54.4% - 71.7%) and specificity of 95.1% (95% CI: 89.1% - 97.9%). ICH: The software identified suspected ICH in 25 of the 26 cases (25/26, 96.1%). Among the 217 ICH-negative cases, the software correctly identified the absence of ICH in 216 (216/217, 99.5%), with one false positive (1217, 0.4%). The overall sensitivity and specificity for ICH detection were 96.2% (95% CI: 81.1% - 99.3%) and 99.5% (95% CI: 97.4% - 99.9%), respectively. ### Comparison with GR and NR readers - LVO detection Eleven total readers independently assessed the presence of LVO. Sensitivities ranged from 20% up to 64.3% with specificities ranging from 70.9% up to 100%. See figure 1 for details of the independent readers in comparison to the platform. Comparison of sensitivities of RAPID and expert readers are shown in Table 1. View this table: [Table 1.](http://medrxiv.org/content/early/2023/08/16/2023.08.11.23294007/T1) Table 1. Sensitivity comparison of GR and NR readers ![Figure 1a:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/16/2023.08.11.23294007/F1.medium.gif) [Figure 1a:](http://medrxiv.org/content/early/2023/08/16/2023.08.11.23294007/F1) Figure 1a: Comparison of the sensitivities with 95% for each reader separated into general and neuroradiologists (blue) and that of RAPID (red). RAPID demonstrates sensitivity higher than all expert readers except one. ![Figure 1b:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/16/2023.08.11.23294007/F2.medium.gif) [Figure 1b:](http://medrxiv.org/content/early/2023/08/16/2023.08.11.23294007/F2) Figure 1b: Comparison of the specificities with 95% for each reader separated into general and neuroradiologists (blue) and that of RAPID (red). RAPID demonstrates a specificity comparable to that of expert readers. ### Diagnostic performance of GR and NR readers - LVO detection ROC analyses measuring diagnostic performance of the readers revealed a wide range in AUC from 0.592 (GR7) to 0.823 (GR3). See figure 2. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/16/2023.08.11.23294007/F3.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2023/08/16/2023.08.11.23294007/F3) Figure 2: ROC curves for individual readers. Using the ratings (1 to 5) scale that readers used to express their confidence in their LVO call we compare reader ROC curves and the overall ROC curves. RAPID uses a decision tree to arrive at classification, there is not an underlying score to generate a ROC curve for RAPID. ### Non-inferiority/superiority testing The primary endpoint was to compare the stand-alone performance of the RAPID NCCT platform to the 11 expert GR and NR readers. We hypothesized that the RAPID NCCT Stroke platform would demonstrate superior performance to the eight GR readers and non-inferior performance to the three NR readers. In assessing non-inferiority of the platform compared to the 11 readers, the platform achieved both non-inferiority as well as superiority, with a significantly higher sensitivity (63.5% versus 43.6%, p < 0.0001).When compared to the eight GR readers only, the platform also showed both non-inferiority and superiority with significantly higher sensitivity (63.5% versus 40.9%, p = 0.001). Non-inferiority and superiority were also established for the comparison with the 3 NR readers alone. Please refer to table 1 for additional detail. The overall agreement with the reference standard was also highest with the software. ### Vessel calcification assessment Of the 244 cases screened, a total of 103 cases were classified as suspected vessel calcifications in the horizontal segment of the MCA. Two cases showed calcification and were classified as true negatives (2/103, 1.9%). An additional 16 true negative cases had calcifications noted elsewhere (16/103, 15.5%). ## Discussion NCCT is a readily available and efficient imaging modality for excluding ICH and detecting early signs of ischemia but CTA is typically required for confirming large vessel occlusions. In this investigation, we focus on the utility of NCCT as it pertains to detecting LVOs and ICH. We report that the RAPID NCCT module has a superior sensitivity for detecting LVO compared with general radiologists and neuroradiologists. The platform also showed excellent sensitivity and specificity in identifying ICH. Our findings suggest that this software has potential utility as an adjunct for neurologists, neurointerventionalists, and radiologists in routine clinical practice for increasing the accuracy of LVO and ICH detection on NCCT. Community hospitals often perform an initial NCCT in patients who present with new neurological symptoms. Subsequently, CTA may be performed if the initial CT does not disclose the diagnosis or if an LVO is suspected. However, CTA acquisition can be challenging in smaller community hospitals and rural centers across the world because of the need for iodinated contrast, concerns for increased radiation exposure, and technical expertise needed to obtain a study that is considered diagnostic8. Furthermore, the delay between clinical assessment/evaluation of the NCCT and when the CTA is obtained can be considerable, often >30 minutes, for the aforementioned reasons. AI software has the potential to address these concerns by substantially reducing this delay when providing an immediate notification that an LVO is suspected. This notification could expedite additional imaging or urgent transfer to a thrombectomy capable center. NCCT is the most common screening modality when stroke is suspected but excluding ICH is essential in order to potentially administer thrombolysis. NCCT is highly sensitive and specific for detecting ICH of all subtypes9. For instance, a meta-analysis by Dubosh et al found NCCT within six hours of symptom onset has a sensitivity and specificity of 98.7% and 99.9% for detection of spontaneous subarachnoid ICH10. The high specificity and sensitivity of NCCT makes it the primary screening modality for not only subarachnoid ICH but all forms of ICH11. Several prior studies have utilized AI platforms in detecting ICH. Goyal used machine learning (ML) techniques in detecting ICH on NCCT with sensitivities ranging from 95-100% and specificities between 85-100%12. Seyam et al similarly used a developed AI based software geared toward ICH detection on NCCT with a sensitivity of 87.2% and specificity of 93.9%13. Others have also utilized deep learning (DL) based models in ICH detection on NCCT with similarly strong results14–16. Software performance for detecting ICH is similar to the Goyal et al and exceeds that of several aforementioned reports, demonstrating a sensitivity of 96.2% and specificity of 99.5%. The Rapid ICH 3 module that is used in the NCCT stroke platform has a sensitivity of 97% and a specificity of 100% for detecting ICHs of any subtype of 0.4 ml or larger17 . When compared to ICH detection, detecting LVO is more challenging on NCCT. The HVS is an important sign of LVO on NCCT evaluation. However, prior studies have reported sensitivities ranging from 17-52% with specificities approaching 100% for HVS detection 3,4,18. Despite the variability in detection, the high specificity of HVS makes it particularly useful in early LVO detection for expeditious transfer to a larger center and/or emergent treatment. Similarly, to ICH evaluation, LVO detection on NCCT using AI applications is a growing area of interest. A systematic review by Shlobin et al concluded that AI applications for LVO detection on CT imaging have reasonable accuracy and show promise as an adjunct tool in the decision-making process19. More recently, a study by Olive-Gadea et al developed a DL technique that identified LVOs on NCCT with 83% sensitivity and 71-85% specificity with human interpretation as the ground truth 20. However, concerns with this algorithm were raised based on the cohort which the algorithm was applied to, the use of clinical and imaging data (NIHSS in addition to NCCT) and the generalizability. The generalizability concern was mainly based on the remote hospital setting the original study performed as the quality of NIHSS assessments may be higher at larger academic institutions 21. In our study, the software detected LVO with a sensitivity of 63.5% and a specificity of 95.1% using only NCCT data. The higher specificity can be particularly useful in smaller centers where mobilizing resources to transfer patients to comprehensive stroke centers may be more challenging. As with previous reports, the software performance was compared to human expert interpreters. We hypothesized that the platform will be superior to that of GR readers and non-inferior to NR readers. Eleven GR and NR readers independently interpreted the cases, showing a sensitivity range of 20-64.3% and specificity of 70.9-100%. When compared to only the eight GR readers, the platform demonstrated superior performance sensitivities (63.5% versus 40.9%, p = 0.001). Furthermore, with the inclusion of NR readers, the platform’s performance was not only non-inferior but still superior with respect to sensitivity (63.5% versus 43.6%, p < 0.001). Our study has several limitations to acknowledge. Firstly, it is limited due to the retrospective design. It is nevertheless strengthened by the robust sample size acquired from five centers that utilize different CT vendors, thus improving generalizability. Secondly, the HVS can be difficult to differentiate from calcifications. We identified cases where calcifications may also be present, all of which were confirmed as true negatives after expert interpretation in order to address this potential limitation. Lastly, although the sensitivity of the platform for independently detecting LVO in absolute measures may be considered low, it is a significant improvement compared to current practice (64% versus 43% for combined GR and NR expert readers). Nevertheless, this study lays the foundation for future investigations exploring the combined sensitivity of the platform with the assessments of interpreting physicians, including neurologists, neurointerventionalists, and radiologists for improved LVO detection. Future studies building upon the current results will also be necessary for detecting posterior circulation and medium vessel occlusions. ## Conclusion The RAPID NCCT Stroke platform demonstrated superior performance to GR and NR readers, suggesting that this software can function as a useful adjunct tool for stroke physicians for expediting the detection of LVOs and urgent transfer to a thrombectomy capable center. Prospective studies are needed for further validation. ## Data Availability Data can be made available upon reasonable request. ## Author Contributions Vivek Yedavalli, analysis and interpretation of data and drafting the manuscript. Greg Albers: design and conceptualization of the study and drafting the manuscript All authors: editing and reviewing. ## Declaration of Interest Drs. Jeremy Heit and Vivek Yedavalli disclose roles as consultants for RAPID (iSchemaView, Menlo Park, CA). Dr. Greg Albers is the co-founder of RAPID. ## Funding No funding source for this investigation. ## Acknowledgements None * Received August 11, 2023. * Revision received August 11, 2023. * Accepted August 16, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. 1. Writing Group Members et al. Heart Disease and Stroke Statistics-2016 Update: A Report From the American Heart Association. Circulation 133, e38–360 (2016). [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjk6IjEzMy80L2UzOCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA4LzE2LzIwMjMuMDguMTEuMjMyOTQwMDcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 2. 2.Macellari, F., Paciaroni, M., Agnelli, G. & Caso, V. Neuroimaging in intracerebral hemorrhage. Stroke 45, 903–908 (2014). [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjQ1LzMvOTAzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTYvMjAyMy4wOC4xMS4yMzI5NDAwNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 3. 3.Potter, C. A. et al. CT for Treatment Selection in Acute Ischemic Stroke: A Code Stroke Primer. Radiographics 39, 1717–1738 (2019). 4. 4.Mair, G. et al. Sensitivity and specificity of the hyperdense artery sign for arterial obstruction in acute ischemic stroke. Stroke 46, 102–107 (2015). [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjQ2LzEvMTAyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTYvMjAyMy4wOC4xMS4yMzI5NDAwNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 5. 5.Sacco, R. L. et al. An updated definition of stroke for the 21st century: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 44, 2064–2089 (2013). [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ0LzcvMjA2NCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA4LzE2LzIwMjMuMDguMTEuMjMyOTQwMDcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 6. 6.Qiu, W. et al. Machine Learning for Detecting Early Infarction in Acute Stroke with Non-Contrast-enhanced CT. Radiology 294, 638–644 (2020). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F16%2F2023.08.11.23294007.atom) 7. 7.Albers, G. W. et al. Thrombectomy for Stroke at 6 to 16 Hours with Selection by Perfusion Imaging. N. Engl. J. Med. 378, 708–718 (2018). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1713973&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29364767&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F16%2F2023.08.11.23294007.atom) 8. 8.Merino, J. G. & Warach, S. Imaging of acute stroke. Nat. Rev. Neurol. 6, 560–571 (2010). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/nrneurol.2010.129&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20842186&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F16%2F2023.08.11.23294007.atom) 9. 9.Arba, F. et al. Noncontrast Computed Tomography Markers of Cerebral Hemorrhage Expansion: Diagnostic Accuracy Meta-Analysis. Int. J. Stroke 17474930211061639 (2021). 10. 10.Dubosh, N. M., Bellolio, M. F., Rabinstein, A. A. & Edlow, J. A. Sensitivity of Early Brain Computed Tomography to Exclude Aneurysmal Subarachnoid Hemorrhage: A Systematic Review and Meta-Analysis. Stroke 47, 750–755 (2016). [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjQ3LzMvNzUwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTYvMjAyMy4wOC4xMS4yMzI5NDAwNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 11. 11.Heit, J. J., Iv, M. & Wintermark, M. Imaging of Intracranial Hemorrhage. J. Stroke Cerebrovasc. Dis. 19, 11–27 (2017). 12. 12.Goyal, R. Intracerebral Hemorrhage Detection in Computed Tomography Scans Through Cost-Sensitive Machine Learning. Appl. Artif. Intell. 36, 2138126 (2022). 13. 13.Seyam, M. et al. Utilization of Artificial Intelligence-based Intracranial Hemorrhage Detection on Emergent Noncontrast CT Images in Clinical Workflow. Radiol Artif Intell 4, e210168 (2022). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/ryai.210168&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35391777&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F16%2F2023.08.11.23294007.atom) 14. 14.Lee, J. Y., Kim, J. S., Kim, T. Y. & Kim, Y. S. Detection and classification of intracranial haemorrhage on CT images using a novel deep-learning algorithm. Sci. Rep. 10, 20546 (2020). 15. 15.Ye, H. et al. Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. Eur. Radiol. 29, 6191–6201 (2019). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00330-019-06163-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31041565&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F16%2F2023.08.11.23294007.atom) 16. 16.Heit, J. J. et al. Automated Cerebral Hemorrhage Detection Using RAPID. AJNR Am. J. Neuroradiol. 42, 273–278 (2021). [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo4OiI0Mi8yLzI3MyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzA4LzE2LzIwMjMuMDguMTEuMjMyOTQwMDcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 17. 17.510(k) Premarket Notification. FDA [https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K221456](https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K221456). 18. 18.Leys, D., Pruvo, J. P., Godefroy, O., Rondepierre, P. & Leclerc, X. Prevalence and significance of hyperdense middle cerebral artery in acute stroke. Stroke 23, 317–324 (1992). [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjIzLzMvMzE3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTYvMjAyMy4wOC4xMS4yMzI5NDAwNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 19. 19.Shlobin, N. A. et al. Artificial Intelligence for Large-Vessel Occlusion Stroke: A Systematic Review. World Neurosurg. 159, 207–220.e1 (2022). 20. 20.Olive-Gadea, M. et al. Deep Learning Based Software to Identify Large Vessel Occlusion on Noncontrast Computed Tomography. Stroke 51, 3133–3137 (2020). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F16%2F2023.08.11.23294007.atom) 21. 21.Amukotuwa, S. A. & Dehkharghani, S. Letter by Amukotuwa and Dehkharghani Regarding Article, ‘Deep Learning Based Software to Identify Large Vessel Occlusion on Noncontrast Computed Tomography’. Stroke; a journal of cerebral circulation vol. 52 e61–e62 (2021).