When AI plays doctor
Physicians discussed the peril and potential of artificial intelligence taking on medical tasks during Internal Medicine Meeting 2025.
How will artificial intelligence (AI) change medicine? The truth is no one knows, but speakers at Internal Medicine Meeting 2025 offered some hypotheses, highlighting potential benefits and risks.
One particular obstacle to forecasting is the large gap between AI theory and the reality in health care right now, according to James Whitfill, MD, MBA, FACP. “I've never seen such a tension between what types of technological advances we're seeing in the lab, i.e., in vitro, and our inability to realize those gains in vivo, or in practice,” he said.
Dr. Whitfill, the chief transformation officer of HonorHealth in Phoenix, reviewed recent research in this area during the C. Wesley Eisele Lecture, “The Implications of AI in Internal Medicine.”
Among the positive examples are findings that AI can predict cardiovascular risk based on a single chest X-ray, published by Annals of Internal Medicine in March 2024, and that it can use electrocardiograms (ECGs) to identify patients at risk of deterioration in time to reduce their risk of death, published by Nature Medicine in April 2024.
“Computers are able to see things in the images that we don't,” said Dr. Whitfill, noting that one eerie aspect of this research is the lack of explanation for how AI makes these diagnoses or predictions.
“That should make you a little bit nervous,” he said. “If I ask one of my residents to look at an ECG and tell me if a patient is having a [myocardial infarction] and they say, ‘Yes' and I say, ‘Why?’ and they say, ‘Cuz,’ I'll say, ‘You should tell me more.’”
He can't have the same conversation with these AI tools. However, AI's ability to communicate is one of the other areas where it's recently reached into medicine. “Six years ago, when I would give a talk about artificial intelligence, I would say, ‘We're never going to see computers be able to be creative, to be empathetic. That's where I'm going to win out as a human physician,’” said Dr. Whitfill.
Yet, at his own organization, AI now helps clinicians draft responses to patient portal messages. “It'll read the incoming message and present a draft so that the physician can edit it,” said Dr. Whitfill. “It certainly seems like that should be beneficial.”
Research on such systems, including small trials published by JAMA Network Open in March and April 2024, has shown that they increase the length of messages from physicians and could appear more empathetic. “These large-language models tend to be much more loquacious,” said Dr. Whitfill.
However, on the outcome that would be most important to the physicians using them, the models did not make a difference. “In the literature to date, there has not been clear improvement in the amount of time physicians were spending, in either reading the patient's message or in writing [a response], so it wasn't a time-saver,” he said.
Likely users would also be disappointed by recent findings about AI's effects on physician burnout, from a study of radiologists, published by JAMA Network Open in November 2024. “Their burnout got worse if they used AI. The AI didn't make their jobs easier, it made their jobs harder,” said Dr. Whitfill.
Highlighting other issues with practicing with AI, an analysis of radiology care, published by the National Bureau of Economic Research in 2023, found that human/computer collaboration was not improving outcomes in this field, concluding that unless changes are made, “optimal human-AI collaboration” will entail assigning cases “to humans or to AI, but rarely to AI assisted humans.”
These conclusions match another study Dr. Whitfill cited. It was published in 2023 by The Lancet Digital Health and found a significantly lower rate of false positives when AI alone interpreted mammograms instead of humans alone or with AI. Overall, readings with AI were noninferior to those by just humans, the study concluded.
“I am not ready to say that we should replace mammographers, particularly my wife,” who is a retired mammographer, he said. “But this is concerning data.”
There are several barriers to humans collaborating effectively with AI, he noted. One is called automation neglect. “When the machine tells me the right answer most of the time, I start to just always agree with it,” he said. The opposite reaction—aversion—results in never listening to the computer's advice.
Even without either of those overreactions, it can be difficult to work AI effectively into humans' workflows, perhaps harder than teaching AI the tasks at hand, he noted.
“My point isn't that we should throw AI away. We just need to study it a lot more,” said Dr. Whitfill. “I believe that the scientific method will tell us what is working and what is not working. Rather than getting caught up in the hype of what these tools might be able to do or what we want them to do or what they should do, we need to be measuring what they actually do for us, for our patients, and our society as a whole.”
Additional cautions were offered during an Internal Medicine Meeting 2025 session called “Medicine and Artificial Intelligence: A Primer for the Practicing Physician.” Matthew Sakumoto, MD, FACP, urged attendees to maintain reasonable skepticism and vigilance when using AI in medicine and avoid the equivalent of “blindly following Google Maps off a cliff.”
He and session director Deepti Pandita, MD, FACP, offered several rules for safer implementation of AI in medical practice.
“First and foremost, do not put any patient health information into publicly available AI software,” said Dr. Sakumoto, a primary care physician and clinical informaticist with Sutter Health in San Francisco. “Even if it's software that you download that says it's HIPAA compliant, it is constantly learning off the information that you put into it.”
If a practice is going to put data into an AI tool, there should be an agreement with the vendor about the future use and protection of the data, added Dr. Pandita, chief medical information officer at the University of California, Irvine.
“You need to have data protection practices evaluated legally,” she said. “If the company says they keep the data forever because machine learning is improving our processes, that may not be acceptable. You may want to cut it down.”
Patients should also have a chance to give consent, according to Dr. Sakumoto, who offered an example of how to do that. Physicians could say, “In our clinic, we use a tool that records our conversation and makes a note of that. Are you OK if we do that?” It's not necessary to specifically say the tool is “AI,” he noted. “That does scare patients.”
Dr. Pandita noted that her health system alerts patients when a message was generated with the help of AI.
Finally, she warned the audience about the importance of checking AI's work, noting that it's very convenient to have it automatically generate patient instructions, especially in a foreign language, but someone still has to review the documents for accuracy.
“CMS regulation still says that anything we hand the patient that's translated needs to be evaluated by a human translator,” she said, adding that a good tool might get information 80% accurate. “That 20% can get you in trouble,” Dr. Pandita said. “I am a proponent for always having a human in the loop.”
Perhaps the most surprising example of how AI might take human physicians out of the loop was offered during the “Hot Topics in Ethics” session. Kyle E. Karches, MD, PhD, FACP, discussed the idea of AI making end-of-life decisions for patients.
The possibility was raised by a Viewpoint published in JAMA Internal Medicine in July 2024. “A critically ill patient who lacks decision-making capacity requires decisions to be made about life-saving treatment, and the patient's family members had never discussed the patient's goals and values,” explained Dr. Karches, associate professor of internal medicine and health care ethics at Saint Louis University.
What if an AI algorithm could crunch the patient's entire digital history—medical records, purchase histories, online conversations—to come up with a prediction about what the patient would have wanted? “That's what the authors suggest,” said Dr. Karches, who is a member of ACP's Ethics, Professionalism and Human Rights Committee, which sponsored the session.
He noted there are many shortcomings to having human surrogates make such decisions, including that they often get them wrong, that they feel a lot of stress about them, and that some patients don't have anyone in their lives to make such decisions.
However, Dr. Karches envisions even more problems with getting AI involved. “AI purports to be more rational,” he said. “In actual human affairs, rationality can't be separated from emotions, motivations, and memories, nor should we want it to be. … We do not simply apply but constantly reinterpret our previous experiences, memories, and convictions.”
He offered the example of a patient who had previously expressed unwillingness to undergo chronic dialysis in an advance directive being glad that her family, who understood the context of her decision, had chosen for her to get temporary dialysis while she was incapacitated.
“This process is often stressful and difficult, but I would argue that it should be,” Dr. Karches said. “It can be a way for surrogates and physicians to honor and care for the patient.”
His final example was a patient who was not planning to undergo additional chemotherapy until she remembered that her granddaughter would be participating in a religious ceremony that was important to her in a few months.
“She reverses her initial impulse based on this judgment about what is relevant. Would AI be capable of such insight?” said Dr. Karches. “I suspect not, but I suspect such a judgment might be available to surrogates and perhaps even to physicians, who are, after all, human, too … We should be wary about dehumanizing medicine any further.”
