Early testing shows concerning results as artificial intelligence makes its way into healthcare settings, with doctors warning about potentially dangerous outcomes for patients.
Recent evaluations by medical professionals reveal that AI systems frequently provide incorrect or potentially harmful medical advice. At Stanford Medical, Clinical Professor Christopher Sharp found AI recommending inappropriate treatments, including potentially dangerous steroid cream applications for sensitive areas like lips.
In another troubling example, when tested with a question about mastitis in breastfeeding mothers, ChatGPT provided recommendations that directly contradicted current medical guidelines from the Academy of Breastfeeding Medicine.
A comprehensive evaluation involving 80 medical professionals and computer scientists discovered that ChatGPT gave dangerous medical responses 20% of the time. "Twenty percent problematic responses is not good enough for actual daily use in the health care system," noted Dr. Roxana Daneshjou, a Stanford medical and data science professor.
Even in more basic applications like medical transcription, AI shows concerning flaws. Doctors report finding completely fabricated information in AI-generated transcripts, including false patient statements and biased assumptions about patients' occupations based on their ethnicity.
While some argue that AI could serve as a tool to support doctors rather than replace them, experts question whether constant verification of AI outputs actually saves time. Dr. Adam Rodman, an internal medicine doctor at Beth Israel Deaconess Medical Center, expresses concern about "hallucinated 'AI slop'" making its way into critical patient care decisions.
The stakes in healthcare are particularly high compared to other AI applications. Unlike a minor error in a PowerPoint presentation, mistakes in medical care can have life-threatening consequences. As AI continues to enter medical settings, patients may want to discuss with their healthcare providers how these tools are being used in their care.