Would You See an AI Doctor?

The term artificial intelligence (AI) has been in use for many years. Indeed, it includes many names, such as AI, neural networks, machine learning and a few other buzzwords. The preceding linked article from IBM tries to explain the meaning of all these terms. A few years ago, IBM made headlines as the developer of Watson, which famously competed on the game show Jeopardy, beating Ken Jennings. There is a new AI interface on the Internet that anyone can try out called ChatGPT. ChatGPT is a language-based AI chatbot that can do many things humans can do, including correcting computer code, writing college entrance essays and a host of other things. A few weeks ago it passed the U.S. Medical Licensing Examination (USMLE), which is required of all physicians who wish to practice in the United States.

“The results ultimately show that large linguistic models, on which ChatGPT was trained, have the potential to assist in medical training and even clinical decision-making,” posited Tiffany H. Kung, MD, a resident tutor at the Harvard School of Medicine, Boston, Massachusetts, and her colleagues, who investigated the performance of ChatGPT in the USMLE in their study.

ChatGPT is in the testing stage of release. Allowing public access may help work out bugs but also raise awareness for marketing purposes. I expect a physician decision-support prototype at some point in the future. The immense medical literature and machine learning from trillions of patient data points could be used to train ChatGPT in medical science and potentially provide medical advice to rival your physician.

The chatbot passed the microbiology quiz designed by microbiologist Alex Berezow, PhD, with flying colors. According to Berezow, the 10 questions were suitable for a college-level final examination. At Stanford University, a not insignificant proportion of students utilized ChatGPT for their final examinations.

For example, one of the questions from Berezow’s microbiology quiz was the following: “A patient presents at the emergency room with a severe headache and stiff neck. The physician prescribes a lumbar puncture to collect cerebrospinal fluid. A Gram stain of the cerebrospinal fluid shows the presence of Gram-negative diplococci. What is the diagnosis?”

ChatGPT answered correctly: “On the basis of the information you provided, the Gram stain of the cerebrospinal fluid shows the presence of Gram-negative diplococci, which are bacteria that are typically oval-shaped and occur in pairs. This finding is consistent with the diagnosis of meningitis.”

About 70% of the population has searched online for medical information. Many do it multiple times a year. Beyond Dr. Google, medical decision-support tools have existed for some time. The VA uses a decision-support tool for its nurse hotline. After my father’s death in 2007 I tried it out and entered my father’s symptoms. It replied with his actual diagnosis, aneurism. Some physicians use decision-support software and nurse practitioners often use it as well. Radiologists sometimes use computer-aided detection (CAD) to interpret mammograms as a backup to human interpretation. Using both a radiologist and CAD together increases accuracy.

The technology is in the preliminary stages but I assume a medical ChatGPT would need to be trained more extensively on medical literature than the current version. Most of the academic journal articles in medicine are gated so I wonder how much of the medical literature the beta version has seen beyond the rudimentary consumer websites hosted by hospitals. Again, more from Medscape Medical News:

One significant limitation is that it is not currently possible to know from which sources the AI draws when formulating is specific response, said Ute Schmid, PhD, at the event ChatGPT and Other Linguistic Models: Between Hype and Controversy. Schmid leads the Cognitive Systems Working Group at the Faculty of Computer Science at the University of Bamberg, Germany.

In Kleesiek’s opinion, and using the example of a medical report, because of its limitations, the linguistic model presents the following challenges:

• Facts must be presented reliably and concisely.

• For patient safety, the suggested medication and dosage must be correct.

• The use of ChatGPT must save time in the composition and must be well integrated into the workflow.

• Questions on liability, data protection, and copyright must be resolved.

It is true that if trained on pseudoscience and quackery, AI will respond in kind (as will humans trained in quackery). This from Stat News:

[W]e began exploring potential medical applications for ChatGPT, which was trained on more than 570 gigabytes of online textual data, extracted from sources like books, web texts, Wikipedia, articles, and other content on the internet, including some focused on medicine and health care. Although the potential usage of AI such as ChatGPT for medical applications excites us, inaccuracies, confabulation, and bias make us hesitant to endorse its use outside of certain situations. These include streamlining education and administrative tasks to assisting clinical decision-making, though even there the application has significant problems and pitfalls.

As one would expect, the medical industrial complex is skeptical of anything that challenges the status quo.

The use of ChatGPT in clinical medicine should be approached with greater caution than its promise in educational and administrative work. In clinical practice, ChatGPT could streamline the documentation process, generating medical charts, progress notes, and discharge instructions. Jeremy Faust, an emergency medicine physician at Brigham and Women’s Hospital in Boston, for instance, put ChatGPT to the test by requesting a chart for a fictional patient with a cough, to which the system responded with a template that Faust remarked was “eerily good.” The potential is obvious: helping health care workers sort through a set of symptoms, determine treatment dosages, recommend a course of action, and the like. But the risk is significant.

So here is the bottom line. At the moment ChatGPT may only be a curiosity but ChatGPT_MD or something like it is likely in the next few years. AI systems will get very good at diagnosing simple conditions and at some point may be able to provide alternative diagnostic information with probabilities and sources. Should powerful tools like this only be available to your doctor? Or should consumer versions be allowed for home use? If they’re allowed for home use should a doctor be the gatekeeper to care? At what point should ChatGPT_MD be allowed to order tests and prescriptions for simple things but alert your physician for certain more serious conditions?

These are all important questions that need to be asked. AI has the potential to revolutionize personalized medicine but the status quo stakeholders won’t give up their coveted positions without a fight.