Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on Performance
Jamil S. Samaan, View ORCID ProfileSamuel Margolis, Nitin Srinivasan, Apoorva Srinivasan, Yee Hui Yeo, Rajsavi Anand, Fadi S. Samaan, James Mirocha, Seyed Amir Ahmad Safavi-Naini, Bara El Kurdi, Ali Soroush, Rabindra Watson, Srinivas Gaddam, Joann G. Elmore, Brennan M.R. Spiegel, Nicholas P. Tatonetti
doi: https://doi.org/10.1101/2024.07.27.24310809
Jamil S. Samaan
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MDSamuel Margolis
2David Geffen School Medicine University of California, Los Angeles, 10833 Le Conte Ave, Los Angeles, CA 90095
BSNitin Srinivasan
3Keck School of Medicine of USC, Health Care Consultation Center, 1510 San Pablo St #514, Los Angeles, CA 90033
BAApoorva Srinivasan
4Department of Computational Biomedicine, Cedars-Sinai Medical Center, West Hollywood, California, USA
MSYee Hui Yeo
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MD, MScRajsavi Anand
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MDFadi S. Samaan
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MSJames Mirocha
5Cedars-Sinai Medical Center, Biostatistics and Bioinformatics Research Center, Los Angeles, CA, USA
MSSeyed Amir Ahmad Safavi-Naini
6Division of Data-Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, NY USA
MDBara El Kurdi
7Division of Gastroenterology and Hepatology, Department of Medicine, Virginia Tech Carilion School of Medicine, Roanoke, VA, USA
MDAli Soroush
6Division of Data-Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, NY USA
8Henry D. Janowitz Division of Gastroenterology, Icahn School of Medicine at Mount Sinai, New York, NY USA
MD, MSRabindra Watson
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MDSrinivas Gaddam
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MD, MPHJoann G. Elmore
2David Geffen School Medicine University of California, Los Angeles, 10833 Le Conte Ave, Los Angeles, CA 90095
MD, MPHBrennan M.R. Spiegel
1Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048
MD, MSHSNicholas P. Tatonetti
4Department of Computational Biomedicine, Cedars-Sinai Medical Center, West Hollywood, California, USA
9Cedars-Sinai Cancer, Cedars-Sinai Medical Center, 8700 Beverly Blvd. Los Angeles, CA, USA
10Department of Biomedical Informatics, Columbia University, New York, New York, USA
PhD![Loading Loading](https://www.medrxiv.org/sites/all/modules/contrib/panels_ajax_tab/images/loading.gif)
- Supplementary Figure 1-6; Supplementary Methods, Supplementary Table 1[supplements/310809_file02.pdf]
Posted July 29, 2024.
Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on Performance
Jamil S. Samaan, Samuel Margolis, Nitin Srinivasan, Apoorva Srinivasan, Yee Hui Yeo, Rajsavi Anand, Fadi S. Samaan, James Mirocha, Seyed Amir Ahmad Safavi-Naini, Bara El Kurdi, Ali Soroush, Rabindra Watson, Srinivas Gaddam, Joann G. Elmore, Brennan M.R. Spiegel, Nicholas P. Tatonetti
medRxiv 2024.07.27.24310809; doi: https://doi.org/10.1101/2024.07.27.24310809
Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on Performance
Jamil S. Samaan, Samuel Margolis, Nitin Srinivasan, Apoorva Srinivasan, Yee Hui Yeo, Rajsavi Anand, Fadi S. Samaan, James Mirocha, Seyed Amir Ahmad Safavi-Naini, Bara El Kurdi, Ali Soroush, Rabindra Watson, Srinivas Gaddam, Joann G. Elmore, Brennan M.R. Spiegel, Nicholas P. Tatonetti
medRxiv 2024.07.27.24310809; doi: https://doi.org/10.1101/2024.07.27.24310809
Subject Area
Subject Areas
- Addiction Medicine (349)
- Allergy and Immunology (668)
- Anesthesia (181)
- Cardiovascular Medicine (2649)
- Dermatology (223)
- Emergency Medicine (399)
- Epidemiology (12229)
- Forensic Medicine (10)
- Gastroenterology (760)
- Genetic and Genomic Medicine (4105)
- Geriatric Medicine (387)
- Health Economics (680)
- Health Informatics (2658)
- Health Policy (1005)
- Hematology (363)
- HIV/AIDS (851)
- Medical Education (399)
- Medical Ethics (109)
- Nephrology (438)
- Neurology (3884)
- Nursing (209)
- Nutrition (578)
- Oncology (2033)
- Ophthalmology (585)
- Orthopedics (241)
- Otolaryngology (306)
- Pain Medicine (250)
- Palliative Medicine (75)
- Pathology (473)
- Pediatrics (1115)
- Primary Care Research (452)
- Public and Global Health (6531)
- Radiology and Imaging (1405)
- Respiratory Medicine (871)
- Rheumatology (410)
- Sports Medicine (343)
- Surgery (449)
- Toxicology (53)
- Transplantation (185)
- Urology (165)