“Artificial intelligence (AI) is increasingly used in healthcare. AI-based chatbots can act as automated conversational agents, capable of promoting health and providing education at any time.” – Görtz, 2023

The old-fashioned general practitioner is a vanished breed. This doctor toted his or her black leather doctor bag to home visits, drew up a chair to couch or bedside, took the patient’s temperature and dispensed TLC along with a scrawly prescription slip. This doctor listened to worries and concerns as much as descriptions of symptoms, and patiently answered questions with reassuring words the patient could understand. This picture evokes the equally-extinct Normal Rockwell illustrations on the covers of The Saturday Evening Post.

Today, patients find themselves having to create online identities for their medical group’s virtual message/medical records system. For many, it’s the least frustrating way to communicate their questions to their doctors, because phone calls may lead down a rabbit hole of voice menus and receptionist messages, and who knows how long before a human being returns the call? There may even be a charge if the physician or nurse replies with medical advice. What’s a patient to do?

Well, many patients turn to the seemingly omniscient internet for medical enlightenment when they experience symptoms. Do they have a disease? What are possible treatments? How do the treatments work? Are they even a candidate?

For men worried about prostate cancer, their search may lead them to websites considered authoritative based on their reputation: The Mayo Clinic, Johns Hopkins, The Prostate Cancer Foundation, American Cancer Society, etc. However, a growing number of people turn to the latest internet resource, Artificial Intelligence (AI) chatbots.

Take, for example, an interesting German study by Görtz, et al. His team of researchers optimistically wrote, “AI-based chatbots can act as automated conversational agents, capable of promoting health and providing education at any time.”^[i] The team designed and evaluated a user-friendly medical chatbot (prostate cancer communication assistant (PROSCA)) for provisioning patient information about early detection of prostate cancer…” To test it, they recruited 10 men (ages 49-81), and 9 of them used the tool and completed questionnaires about it. 78% said they needed no assistance using it, so the team judged it to be straightforward. The PROSCA information was evidence-based, and 89% reported a clear to moderate increase in their PCa knowledge, and all said they would re-use a chatbot in the future.

PROSCA was programmed to offer “… targeted information about PC, invasive diagnostics, potential complications as well as treatment options to reduce decisional conflict, enhance patient knowledge, and promote shared decision-making.” In this case, the chatbot was intended to promote physician-patient communication. The situation was ideal, because the PROSCA team was involved with the participants throughout the process. This is not the case for the man spending time on his home or work computer seeking information about prostate cancer. He’s a lone wolf, and with AI chatbots starting to sprout like mushrooms, it’s reminiscent of Davey Crockett exploring the hinterlands of Tennessee.

Two studies published during August, 2023 in JAMA suggest it’s not exactly the AI Wild West out there, but at this stage the reviews are mixed.

• Pan, et al. identified the 5 most searched questions on prostate cancer (also skin, lung, breast and colorectal cancers), and posed them to four AI chatbots to analyze the quality of responses (clarity, actionability, misinformation, readability). They agreed there was no misinformation, but concluded that “AI chatbots generally produce accurate information for the top cancer related search queries, but the responses are not readily actionable and are written at a college reading level. These limitations suggest that AI chatbots should be used supplementarily and not as a primary source for medical information.”^[ii]
• Chen, et al. compared one particular chatbot vs. the 2021 National Comprehensive Cancer Network guidelines regarding treatment strategies for prostate cancer and others. Three cancer specialists looked for side-by-side agreement between the chatbot’s responses and NCCN guidelines. 62% of the time, the three specialists agreed on how they scored agreement, but when they couldn’t agree it was because the chatbot’s responses were unclear. Also, 13 out of 104 responses were “hallucinated,” by which the authors meant “primarily recommendations for localized treatment of advanced disease, targeted therapy, or immunotherapy.”^[iii] A Medscape news story summed it up: Based on the findings, the team recommended that clinicians advise patients that AI chatbots are not a reliable source of cancer treatment information.

There is no doubt that a) PCa patients will continue to use the internet for information/education, and b) it’s reasonable to expect that chatbot information will improve over time. Even so, there is no substitute for the doctor-patient relationship in real time, whether in person or via telemedicine. No chatbot will ever know you as your doctor does. The “cocktail” of interactive factors (e.g., eye contact, voice tone, gestures, smiles, etc.) that comprise communication between persons can’t be programmed into a chatbot (at least, not in today’s world, but maybe in science fiction).

We agree with the recommendation that online information be used as a supplement to, not a substitute for, medical opinion and advice.

NOTE: This content is solely for purposes of information and does not substitute for diagnostic or medical advice. Talk to your doctor if you are experiencing pelvic pain, or have any other health concerns or questions of a personal medical nature.

[i] Görtz M, Baumgärtner K, Schmid T, Muschko M, Woessner P, Gerlach A, Byczkowski M, Sültmann H, Duensing S, Hohenfellner M. An artificial intelligence-based chatbot for prostate cancer education: Design and patient evaluation study. Digit Health. 2023 May 2;9:20552076231173304.
[ii] Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About Cancer. JAMA Oncol. 2023 Aug 24:e232947.
[iii] Chen S, Kann BH, Foote MB, Aerts HJWL et al. Use of Artificial Intelligence Chatbots for Cancer Treatment Information. JAMA Oncol. 2023 Aug 24:e232954.