AI’s ChatGPT Faces Challenges in Medical Examinations
In a series of studies, ChatGPT, the renowned conversational AI model, has been put to the test in various medical fields. While it managed to pass the United States Medical Licensing Exam, it later stumbled in ophthalmology, gastroenterology, and urology exams. The results have sparked a debate about the efficacy and potential applications of AI in medical diagnostics and education.
- ChatGPT passed the USMLE with over 50% accuracy, exceeding 60% in some analyses.
- In ophthalmology, it failed a practice exam twice but excelled in real-world scenarios.
- ChatGPT-3 and ChatGPT-4 both failed the American College of Gastroenterology self-assessment test.
- In urology, it answered only 26.7% of open-ended questions and 28.2% of multiple-choice questions correctly.
- Concerns have been raised about the potential dissemination of inaccurate medical information if ChatGPT is not monitored or regulated.
- ChatGPT’s scores improved with new versions, and researchers suggest it may help create the USMLE in the future.
- The quality of information provided by ChatGPT in gastroenterology was found to be contingent upon the quality of the training data.
- Some argue that traditional testing may not be the best way to measure AI’s intelligence or efficacy.
- Although ChatGPT has shown promise in some areas, its inconsistent performance in medical exams highlights the need for further development and careful consideration of its role in healthcare.
Did You Know?
Artificial intelligence in healthcare is projected to grow at an exponential rate, with the potential to reach more than $34 billion by 2025.