Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLE

Yahya Shaikh; Zainab Asiya; Muzamila Mushtaq Jeelani; Aamir Javaid; Tauhid Mahmud; Shiv Gaglani; Michael Christopher Gibbons; Minahil Cheema; Amanda Cross; Denisa Livingston; Elahe Nezami; Ronald Dixon; Ashwini Niranjan-Azadi; Saad Zafar; Zishan Siddiqui

doi:10.1101/2025.02.17.25322388

Data Availability

All relevant data are within the manuscript and its Supporting Information files.

https://github.com/councilofai/project-saru

Posted February 20, 2025.

Download PDF

Author Declarations

Data/Code

Citation Tools

Get QR code

Tweet Widget

Subject Area

Health Informatics

Reviews and Context

Comment

TRIP Peer Reviews

Community Reviews

Automated Services

Blogs/Media

Author Videos

Subject Areas

All Articles

Addiction Medicine (425)
Allergy and Immunology (746)
Anesthesia (219)
Cardiovascular Medicine (3240)
Dentistry and Oral Medicine (358)
Dermatology (271)
Emergency Medicine (476)
Endocrinology (including Diabetes Mellitus and Metabolic Disease) (1153)
Epidemiology (13263)
Forensic Medicine (19)
Gastroenterology (892)
Genetic and Genomic Medicine (5077)
Geriatric Medicine (471)
Health Economics (776)
Health Informatics (3203)
Health Policy (1130)
Health Systems and Quality Improvement (1177)
Hematology (423)
HIV/AIDS (1005)
Infectious Diseases (except HIV/AIDS) (14557)
Intensive Care and Critical Care Medicine (903)
Medical Education (468)
Medical Ethics (126)
Nephrology (515)
Neurology (4829)
Nursing (256)
Nutrition (717)
Obstetrics and Gynecology (871)
Occupational and Environmental Health (783)
Oncology (2490)
Ophthalmology (709)
Orthopedics (279)
Otolaryngology (338)
Pain Medicine (321)
Palliative Medicine (89)
Pathology (533)
Pediatrics (1284)
Pharmacology and Therapeutics (543)
Primary Care Research (551)
Psychiatry and Clinical Psychology (4137)
Public and Global Health (7397)
Radiology and Imaging (1679)
Rehabilitation Medicine and Physical Therapy (995)
Respiratory Medicine (975)
Rheumatology (474)
Sexual and Reproductive Health (492)
Sports Medicine (417)
Surgery (536)
Toxicology (70)
Transplantation (233)
Urology (202)

Comments

medRxiv aims to provide a venue for anyone to comment on a medRxiv preprint. Comments are moderated for offensive or irrelevant content (this can take ~24 h). Please avoid duplicate submissions and read our Comment Policy before commenting. The content of a comment is not endorsed by medRxiv.

medRxiv aims to inform readers about online discussion of this preprint occurring elsewhere. The content at the links below is not endorsed by either medRxiv or the preprint's authors.

Community reviews for this article:

There are no community reviews for this paper.

Automated Evaluations

Certain services provide automated analysis of preprints. Analyses invited by the authors are displayed at the top of this tab. Those done independently of authors are shown underneath . None of these analyses is endorsed by medRxiv.

Automated Evaluations:

There are no automated evaluations for this paper.

Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLE

Data Availability

Subject Area

Follow this preprint