Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models
Tiffany H. Kung, Morgan Cheatham, ChatGPT, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, Victor Tseng
doi: https://doi.org/10.1101/2022.12.19.22283643
Tiffany H. Kung
1AnsibleHealth, Inc (Mountain View, CA)
2Department of Anesthesiology, Massachusetts General Hospital, Harvard School of Medicine (Boston, MA)
Morgan Cheatham
3Warren Alpert Medical School; Brown University (Providence, RI)
4OpenAI, Inc; (San Francisco, CA)
Arielle Medenilla
1AnsibleHealth, Inc (Mountain View, CA)
Czarina Sillos
1AnsibleHealth, Inc (Mountain View, CA)
Lorie De Leon
1AnsibleHealth, Inc (Mountain View, CA)
Camille Elepaño
1AnsibleHealth, Inc (Mountain View, CA)
Maria Madriaga
1AnsibleHealth, Inc (Mountain View, CA)
Rimel Aggabao
1AnsibleHealth, Inc (Mountain View, CA)
Giezel Diaz-Candido
1AnsibleHealth, Inc (Mountain View, CA)
James Maningo
1AnsibleHealth, Inc (Mountain View, CA)
Victor Tseng
1AnsibleHealth, Inc (Mountain View, CA)
5Department of Medical Education, UWorld, LLC (Dallas, TX)
Data Availability
The data analyzed in this study were obtained from USMLE sample questions sets which are publicly available. The question index, raw inputs, and raw AI outputs are available in the Online Data Supplement. Inquiries and requests for additional dataset items and adjudication results can be provided upon reasonable request by contacting Victor Tseng, MD (victor{at}ansiblehealth.com).
Posted December 21, 2022.
Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models
Tiffany H. Kung, Morgan Cheatham, ChatGPT, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, Victor Tseng
medRxiv 2022.12.19.22283643; doi: https://doi.org/10.1101/2022.12.19.22283643
Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models
Tiffany H. Kung, Morgan Cheatham, ChatGPT, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, Victor Tseng
medRxiv 2022.12.19.22283643; doi: https://doi.org/10.1101/2022.12.19.22283643
Subject Area
Subject Areas
- Addiction Medicine (399)
- Allergy and Immunology (710)
- Anesthesia (201)
- Cardiovascular Medicine (2952)
- Dermatology (250)
- Emergency Medicine (440)
- Epidemiology (12758)
- Forensic Medicine (12)
- Gastroenterology (829)
- Genetic and Genomic Medicine (4593)
- Geriatric Medicine (420)
- Health Economics (729)
- Health Informatics (2923)
- Health Policy (1069)
- Hematology (389)
- HIV/AIDS (925)
- Medical Education (427)
- Medical Ethics (116)
- Nephrology (469)
- Neurology (4366)
- Nursing (237)
- Nutrition (640)
- Oncology (2274)
- Ophthalmology (647)
- Orthopedics (258)
- Otolaryngology (325)
- Pain Medicine (279)
- Palliative Medicine (83)
- Pathology (501)
- Pediatrics (1197)
- Primary Care Research (499)
- Public and Global Health (6949)
- Radiology and Imaging (1531)
- Respiratory Medicine (915)
- Rheumatology (439)
- Sports Medicine (385)
- Surgery (490)
- Toxicology (60)
- Transplantation (212)
- Urology (181)