Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on Performance
Jamil S. Samaan MD, View ORCID ProfileSamuel Margolis BS, Nitin Srinivasan BA, Apoorva Srinivasan MS, Yee Hui Yeo MD, MSc, Rajsavi Anand MD, Fadi S. Samaan MS, James Mirocha MS, Seyed Amir Ahmad Safavi-Naini MD, Bara El Kurdi MD, Ali Soroush MD, MS, Rabindra Watson MD, Srinivas Gaddam MD, MPH, Joann G. Elmore MD, MPH, Brennan M.R. Spiegel MD, MSHS, Nicholas P. Tatonetti PhD
doi: https://doi.org/10.1101/2024.07.27.24310809
Jamil S. Samaan MD
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
Samuel Margolis BS
2 UCLA;
Nitin Srinivasan BA
3 Keck School of Medicine of USC, Health Care Consultation Center, 1510 San Pablo St #514, Los Angeles, CA 90033.;
Apoorva Srinivasan MS
4 Department of Computational Biomedicine, Cedars-Sinai Medical Center, West Hollywood, California, USA.;
Yee Hui Yeo MD, MSc
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
Rajsavi Anand MD
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
Fadi S. Samaan MS
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
James Mirocha MS
5 Cedars-Sinai Medical Center, Biostatistics and Bioinformatics Research Center, Los Angeles, CA, USA.;
Seyed Amir Ahmad Safavi-Naini MD
6 Division of Data-Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, NY USA.;
Bara El Kurdi MD
7 Division of Gastroenterology and Hepatology, Department of Medicine, Virginia Tech Carilion School of Medicine, Roanoke, VA, USA.;
Ali Soroush MD, MS
8 Division of Data-Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, NY USA. Henry D. Janowitz Division of Gastroenterology, Icahn School of Medicine at Mount Sinai, New York, NY USA.;
Rabindra Watson MD
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
Srinivas Gaddam MD, MPH
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
Joann G. Elmore MD, MPH
9 David Geffen School Medicine University of California, Los Angeles, 10833 Le Conte Ave, Los Angeles, CA 90095.;
Brennan M.R. Spiegel MD, MSHS
1 Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048.;
Nicholas P. Tatonetti PhD
10 Department of Computational Biomedicine, Cedars-Sinai Medical Center, West Hollywood, California, USA. Cedars-Sinai Cancer, Cedars-Sinai Medical Center, 8700 Beverly Blvd. Los Angeles, CA, USA. Department of Biomedical Informatics, Columbia University, Ne
![Loading Loading](https://www.medrxiv.org/sites/all/modules/contrib/panels_ajax_tab/images/loading.gif)
Data Availability
2022 American College of Gastroenterology (ACG) self-assessment examination. Available at https://education.gi.org/satest/satest_18
Posted July 29, 2024.
Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on Performance
Jamil S. Samaan MD, Samuel Margolis BS, Nitin Srinivasan BA, Apoorva Srinivasan MS, Yee Hui Yeo MD, MSc, Rajsavi Anand MD, Fadi S. Samaan MS, James Mirocha MS, Seyed Amir Ahmad Safavi-Naini MD, Bara El Kurdi MD, Ali Soroush MD, MS, Rabindra Watson MD, Srinivas Gaddam MD, MPH, Joann G. Elmore MD, MPH, Brennan M.R. Spiegel MD, MSHS, Nicholas P. Tatonetti PhD
medRxiv 2024.07.27.24310809; doi: https://doi.org/10.1101/2024.07.27.24310809
Multimodal Large Language Model Passes Specialty Board Examination and Surpasses Human Test-Taker Scores: A Comparative Analysis Examining the Stepwise Impact of Model Prompting Strategies on Performance
Jamil S. Samaan MD, Samuel Margolis BS, Nitin Srinivasan BA, Apoorva Srinivasan MS, Yee Hui Yeo MD, MSc, Rajsavi Anand MD, Fadi S. Samaan MS, James Mirocha MS, Seyed Amir Ahmad Safavi-Naini MD, Bara El Kurdi MD, Ali Soroush MD, MS, Rabindra Watson MD, Srinivas Gaddam MD, MPH, Joann G. Elmore MD, MPH, Brennan M.R. Spiegel MD, MSHS, Nicholas P. Tatonetti PhD
medRxiv 2024.07.27.24310809; doi: https://doi.org/10.1101/2024.07.27.24310809
Subject Area
Subject Areas
- Addiction Medicine (334)
- Allergy and Immunology (657)
- Anesthesia (177)
- Cardiovascular Medicine (2561)
- Dermatology (217)
- Emergency Medicine (389)
- Epidemiology (12065)
- Forensic Medicine (10)
- Gastroenterology (741)
- Genetic and Genomic Medicine (3980)
- Geriatric Medicine (375)
- Health Economics (666)
- Health Informatics (2569)
- Health Policy (991)
- Hematology (357)
- HIV/AIDS (823)
- Medical Education (394)
- Medical Ethics (106)
- Nephrology (421)
- Neurology (3744)
- Nursing (206)
- Nutrition (558)
- Oncology (1946)
- Ophthalmology (565)
- Orthopedics (233)
- Otolaryngology (299)
- Pain Medicine (246)
- Palliative Medicine (72)
- Pathology (469)
- Pediatrics (1088)
- Primary Care Research (442)
- Public and Global Health (6409)
- Radiology and Imaging (1354)
- Respiratory Medicine (857)
- Rheumatology (393)
- Sports Medicine (336)
- Surgery (431)
- Toxicology (51)
- Transplantation (184)
- Urology (162)