Are AI Models Ready for Medical Exams? Unveiling LLM Performance on Spain's Residency Test

In the rapidly evolving landscape of healthcare, the potential for Large Language Models (LLMs) to assist in clinical decision-making is exciting yet complex. A recent study led by Miguel Fernandez-de-Retana and his team at the University of Deusto explores the capabilities of these models, specifically regarding their performance on the Spanish Médico Interno Residente (MIR) examination. Their findings reveal intriguing insights about the reliance on AI in high-stakes medical settings.

Context: The Spanish Residency Examination

The MIR examination plays a crucial role in determining the future of medical graduates in Spain. It assesses candidates through multiple-choice questions, with penalties for incorrect answers, making it an ideal benchmark to evaluate the reliability and reasoning abilities of AI models. The study scrutinizes whether LLMs can not only perform well but also maintain a sound understanding of their limitations.

The Study's Findings: Top-Performing Models and Their Limitations

The researchers evaluated four LLMs, ranging in size from 8 to 27 billion parameters, simulating the exam conditions with 1,552 questions from the MIR. While the AI models demonstrated promising scores, ranking among the top candidates, the study uncovers significant weaknesses:

• Overconfidence in Predictions: The models often provided confident answers yet exhibited a concerning trend of being wrong for the wrong reasons. Up to 38% of their correct responses were based on dismissing options for clinically unsound reasons.
• Self-Knowledge Poorly Calibrated: Although medical fine-tuning aimed to enhance their performance, it paradoxically reduced their accuracy without improving their ability to distinguish correct from incorrect answers.
• Image Utilization Ineffective: The evaluation also tested how well the models incorporated clinical images into their reasoning. Surprisingly, changing or removing images did not affect the accuracy of their answers.

The Implications for Future AI in Healthcare

This study raises vital questions about the competency of AI models in clinical settings. The evidence reveals a reliance on these models could lead to serious issues if not thoroughly audited and understood. The researchers argue that evaluations of LLMs should go beyond just score metrics to include the models' reasoning capabilities, confidence levels, and the practical application of contextual information such as images.

A Call for Caution

As hospitals consider adopting AI systems for clinical decision support, the insights from this research highlight the necessity for rigorous validation and continuous oversight. It's imperative to ensure that while AI can provide assistance, it should not be viewed as a replacement for the nuanced clinical reasoning that human practitioners offer. The future may rest on a coordinated approach where AI and human expertise combine to enhance patient care.

In summary, while LLMs are making strides in passing medical exams, the journey to reliable, safe AI in medicine is still ongoing. The findings from the study prompt crucial discussions surrounding the integration of AI into healthcare, advocating for a blend of technological advancement and critical human oversight.

Authors: Miguel Fernandez-de-Retana, Unai Zulaika, Rubén Sánchez-Corcuera, Aritz Bilbao-Jayo, Aitor Almeida