A study published in JAMA Network Open examined the use of the artificial intelligence (AI) large language model system GPT-4 by 50 U.S.-licensed physicians in family medicine, internal medicine, and emergency medicine. The researchers found that while GPT-4 alone performed better than clinicians using conventional diagnostic resources, the addition of GPT-4 did not significantly improve clinical reasoning when compared to clinicians using conventional resources alone. These findings highlight the complex nature of integrating AI into clinical practice and suggest that more research is needed to understand how AI can best support healthcare professionals.
Andrew Olson, MD, a professor at the University of Minnesota Medical School, emphasized the importance of studying AI tools and their impact on medical care. He noted that while the field of AI is rapidly expanding and impacting various aspects of our lives, there is still room for improvement in physician-AI collaboration in clinical practice. The study results indicate that there are opportunities for further enhancement in this area, underscoring the need for continued exploration into the use of AI in healthcare.
The researchers found that GPT-4 demonstrated better diagnostic performance when used alone, surpassing the performance of clinicians using conventional diagnostic resources or clinicians assisted by GPT-4. However, the integration of GPT-4 as a diagnostic aid alongside clinicians did not significantly improve clinical reasoning. This nuanced potential for AI in healthcare highlights the importance of further investigation into how AI can effectively support clinical practice and improve patient care.
In response to these findings, the collaborating institutions have launched a bi-coastal AI evaluation network called ARiSE to further evaluate the outputs of GenAI in healthcare. This initiative aims to explore the use of AI tools in clinical settings and assess their impact on diagnosis and treatment. By conducting more studies and gathering data on how clinicians should be trained to use these tools, researchers hope to optimize the integration of AI in healthcare and enhance the overall patient experience.
The study was funded by the Gordon and Betty Moore Foundation, which supports research in the fields of science, environmental conservation, patient care, and the San Francisco Bay Area. The foundation’s contribution enabled the research team to conduct this study and explore the potential benefits of AI in clinical practice. With continued support from organizations like the Gordon and Betty Moore Foundation, researchers can continue to advance the field of AI in medicine and develop innovative solutions to improve patient outcomes and healthcare delivery.