LLM 4.0 from OpenAI answered 85% of the American Board of Psychiatry and Neurology’s questions correctly, implying potentially “significant” future uses for the technology.
OpenAI’s latest update of its large language model (LLM), ChatGPT-4.0, has passed a clinical neurology exam with 85% correct answers in a proof-of-concept study. The research authors believe that after some fine-tuning, LLMs could have “significant applications” in clinical neurology.
The results of the experiment, conducted by a group of researchers from the University Hospital Heidelberg and the German Cancer Research Center in Heidelberg, were published on Dec. 7. The test, performed on May 31, featured two LLMs, ChatGPT-3.5 and its later version, ChatGPT-4.0.
The researchers used the bank of questions for a neurology exam from the American Board of Psychiatry and Neurology with a small cohort of questions from the European Board for Neurology.
Read more