Artificial intelligence applications have been in the news lately. I’m especially interested in medical applications. I’ve previously written about using AI to help radiologists interpret X-rays and diagnostic images. Computer-aided radiology interpretation has been around for a few years and has gotten to the point where AI can catch things that radiologists miss. The New York Times worries about whether AI is ready to manage patient care. The consensus seems to be that although it may be premature at the moment it soon will be.
One complaint is that among the (mostly correct) answers AI will throw out some totally bogus nonsense it learned from the Internet. Besides reputable medical sources, the Internet also features plenty of New Age psychobabble perpetuated by snake oil salesmen who promote their dietary supplements, flimflam therapies and alternative medical nonsense. Perhaps instead of reading the entire Internet med tech AI should exclusively train on medical journals and vetted medical websites edited by experts.
Forward, a med tech start, up just received a $100 million cash infusion. It is a serious effort to radically disrupt the industry, with nearly $700 million in total funding to date. The company is exploring a concept called CarePods, self-service primary care kiosks that use artificial intelligence to diagnose and screen for health conditions. CarePods will be located in malls, office buildings, gyms and areas where people congregate. (I hope CarePods look like Dr. Who’s red British phone booth.)
CarePods are arguably the most exciting investment into primary care that I’ve read about. The firm began in 2016 as a cash-pay, direct primary care model. Side note: I love direct pay primary care. I’ve told the story before about when I was in grad school my doctor left town and had his office refer patients to a colleague who ran a cash pay clinic. This physician accepted no insurance and did not schedule appointments. It was first come, first served. I went to him around 30 years ago and only had to wait 10 minutes for my visit. The cost was $35, but conventional clinics would have needed to charge three times that in order to net a similar profit after paying office staff, insurance billing clerks and overhead for a larger office.
The firm is moving away from brick & mortar clinics to cheaper software and hardware-driven AI kiosks. The CarePods model is for continuous care rather than an episodic, urgent care model. Membership starts at $99 a month. Licensed physicians will review and oversee treatments and write prescriptions. Some medical establishment elites will undoubtedly see this as substandard care that is a threat to their guild. On the other hand, it could be the wave of the future that expands access to care to everyone. The CEO told Fierce Healthcare:
“We ask ourselves, ‘What would it take to get healthcare to the whole planet?’ You quickly realize when you peel back the layers of the onion that we’re doing healthcare all wrong. Today, healthcare is based on doctors and nurses. And, they’re awesome, but you’re never going to scale doctors and nurses to the whole planet. Our insight was actually healthcare should just be a product. We should just take every single thing that doctors and nurses are doing and just migrate it over to hardware and software because we can scale that healthcare to the planet,” Aoun said.
How good could an AI diagnosis be without a doctor (or even a nurse) on site? A recent study found that AI performed better than virtually all human readers in complex medical cases.
OpenAI’s GPT-4 correctly diagnosed 52.7% of complex challenge cases, compared to 36% of medical journal readers, and outperformed 99.98% of simulated human readers, according to a study published by the New England Journal of Medicine.
The evaluation, conducted by researchers in Denmark, utilized GPT-4 to find diagnoses pertaining to 38 complex clinical case challenges with text information published online between January 2017 and January 2023. GPT-4’s responses were compared to 248,614 answers from online medical journal readers.
Each complex clinical case included a medical history alongside a poll with six options for the most likely diagnosis. The prompt used for GPT-4 asked the program to solve for diagnosis by answering a multiple choice question and analyzing full unedited text from the clinical case report. Each case was presented to GPT-4 five times to evaluate reproducibility.
Patients in the clinical cases ranged from newborn to 89 years of age, and 37% were female.
The recent March 2023 edition of GPT-4 correctly diagnosed 21.8 cases or 57% with good reproducibility, while medical journal readers correctly diagnosed 13.7 cases, or 36% on average.
A weakness of the study was that there is no easy way to gauge the expertise of the medical journal readers providing responses. Responses were presumably physicians with knowledge of the specialty but that is by no means certain. However, the wisdom of crowds has been shown to improve accuracy so I would not dismiss crowd-sourced answers. Based on theoretically known limitations, the study authors believe that GPT-4 could outperform nearly three-quarters (72%) of medical journal readers with “maximum correlated correct answers”. I’m not actually sure what the last quote even means, but GPT-4 is apparently very good and will likely get better over time. The future of primary care is artificial intelligence if the medical establishment doesn’t try to block it. A way to leverage CarePods would be to create an online interface for smartphones and computers.