Patients More Satisfied With AI’s Answers Than Those From Their Doctor

Patients were consistently more satisfied with responses from artificial intelligence (AI) to messages in the electronic health record than they were with those from their clinician, according to a study in JAMA Network Open.

In this video interview, author Eleni Linos, MD, MPH, DrPH, of Stanford University in California, discusses the findings and their implications for AI’s future in medicine.

The following is a transcript of her remarks:

The goal of this study was to try and evaluate how satisfied patients are with the responses generated by AI to real-world clinical questions.

So as we know, communication with medical providers and clinicians is a huge part of healthcare in order to answer questions, clarify symptoms, prescriptions, diagnosis — so that level of communication is increasingly becoming more common. What we wanted to understand is whether or not replies to patient messages that are generated by some of these novel generative AI technologies were helpful and useful to patients.

What we found and what surprised us was that when patients evaluated how satisfied they were with these AI-generated responses, they were actually more satisfied with the AI-generated replies than they were with the real clinician replies. And obviously that surprised us, because we always think the gold standard is a doctor replying to a message themselves.

What we found in reality was that a lot of these messages that real doctors write are short, they don’t have as much information because frankly, we’re all pressed for time, and even though we do want to deliver the best care for our patients, often these AI-generated responses are more detailed, are obviously crafted to have empathy, and are longer. It was really interesting to see that patients who evaluated and compared the AI-generated to the real clinician responses found the AI-generated ones more satisfactory.

The reason this comparison is important is because we were trying to show whether or not these AI-generated responses were as good as or good enough to be used in clinical practice. These technologies are already incorporated into clinical care and increasingly will become even more so. Doing rigorous scientific studies to try and understand both the accuracy and the effectiveness of these tools is important.

Until now, there had been really good evidence showing that it saves clinicians time, which has these downstream benefits of possibly improving the physician experience and reducing burnout. So we know it’s helpful for the healthcare providers, but what our study wanted to answer is: How do patients feel about it? Is it helpful for patients?

My recommendation would be that we continue doing innovative implementation of these techniques in real-world healthcare settings, and as we’re implementing we are paying really close attention to evaluate how these interventions and these pilots that we’re trying in real healthcare settings, how they’re making patients feel and also how they are impacting the care patients receive. So it’s really important as technology is moving at this incredibly rapid pace that researchers don’t delay the implementation of technology, but they are right there by the side of all of the leaders implementing these tools evaluating and studying how these tools are helping patients, helping physicians, and measuring these impacts as we go.

We’re super excited at the Stanford Center for Digital Health that we’re launching our first annual symposium next week. We’re going to be live streaming the keynote talks and would love all of you to join and listen in on the discussions that we’ll be having on this and many other topics relevant to digital health.

Please enable JavaScript to view the comments powered by Disqus.