PT  - JOURNAL ARTICLE
AU  - Sun, Dongmei
AU  - Nguyen, Thanh M.
AU  - Allaway, Robert J.
AU  - Wang, Jelai
AU  - Chung, Verena
AU  - Yu, Thomas V
AU  - Mason, Michael
AU  - Dimitrovsky, Isaac
AU  - Ericson, Lars
AU  - Li, Hongyang
AU  - Guan, Yuanfang
AU  - Israel, Ariel
AU  - Olar, Alex
AU  - Pataki, Balint Armin
AU - RA2 DREAM Challenge Community
AU  - Stolovitzky, Gustavo
AU  - Guinney, Justin
AU  - Gulko, Percio S.
AU  - Frazier, Mason B.
AU  - Costello, James C.
AU  - Chen, Jake Y.
AU  - Bridges, S. Louis
TI  - A Crowdsourcing Approach to Develop Machine Learning Models to Quantify Radiographic Joint Damage in Rheumatoid Arthritis
AID  - 10.1101/2021.10.25.21265495
DP  - 2021 Jan 01
TA  - medRxiv
PG  - 2021.10.25.21265495
4099  - http://medrxiv.org/content/early/2021/10/26/2021.10.25.21265495.short
4100  - http://medrxiv.org/content/early/2021/10/26/2021.10.25.21265495.full
AB  - To develop machine learning methods to quantify joint damage in patients with rheumatoid arthritis (RA), we developed the RA2 DREAM Challenge, a crowdsourced competition that utilized existing radiographic images and “gold-standard” scores on 674 sets of films from 562 patients. Training and leaderboard sets were provided to participants to develop methods to quantify joint space narrowing and erosions. In the final round, participants submitted containerized codes on a test set; algorithms were evaluated using weighted root mean square error (RMSE). In the leaderboard round, there were 173 submissions from 26 teams in 7 countries. Of the 13 submissions in the final round, four top-performing teams were identified. Robustness of results was assessed using Bayes factor and validated using an independent set of radiographs. The top-performing algorithms, which consisted of different styles of deep learning models, provided accurate and robust quantification of joint damage in RA. Ultimately, these methods lay the groundwork to accelerate research and help clinicians to optimize treatments to minimize joint damage.Competing Interest StatementThe following authors have financial interests in these companies: YG, Genentech, F. Hoffmann-La Roche AG, Lilly, Merck KGaA, Ann Arbor Algorithms Inc; MM, Bristol-Myers Squibb; PG, Oryn Therapeutics, CannBioRx, Gilead; JCC, PrecisionProfile. All other authors have no competing interests to disclose.Funding StatementWe would like to acknowledge funding support from Bristol Myers Squibb (BMS) and the CTSA Program National Center for Data to Health (CD2H) is supported by the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (Grant U24TR002306).Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:the UAB Institutional Review Board for Human Use gave ethical approval for this workI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll radiogrpahic images used in this study are available upon reasonable request to the authors. Method write-ups and Docker containers for all submitted methods are available from the Challenge workspace (www.doi.org/10.7303/syn20545111). https://www.synapse.org/#!Synapse:syn20545111/wiki/594083