When patients ask AI about you, what wrong answer are they getting?

A patient finds out they need thyroid surgery. They search for information about the procedure at their local hospital. They want to know the basics: what it costs, whether their insurance is accepted, how long recovery takes. They don't call the hospital or visit the website. They ask their favorite artificial intelligence (AI) program.

And AI answers. Confidently and completely, except in at least some of what it says, incorrectly.

This is not a hypothetical. It happens thousands of times a day across every type of health care organization in the country, and most of those organizations have no idea what AI is saying about them.

The conversation we are not having

Most of the health care industry's anxiety about AI hallucinations focuses on what AI gets wrong. The fabricated citations. The invented drug interactions. The confident wrong answers about clinical protocols that sound entirely plausible until a clinician catches them.

That is a real problem. Research from Blueline Insights shows that 91.8% of clinicians have already encountered AI hallucinations in a health care context. That number should stop people in their tracks.

But the more urgent problem, and the one that gets far less attention, is the hallucinations that happen not because AI invents something, but because your organization never said anything at all.

AI systems are built to answer questions. When a patient is trying to decide whether to call your practice, whether to trust your doctors or whether your hospital is even the right place for their diagnosis, AI will confidently produce an answer. If your website provides a clear and accurate answer, AI will find it. If it does not, AI will find something else, whether it be a competitor's page or a Reddit post from three years ago. Or even worse, it will construct something that sounds reasonable and present it as fact.

I recently searched "How much does Healthcare Success cost" and found that most AI engines were getting it wrong, with some saying we accept very small budgets appropriate for single-practice doctors. So we made changes, especially to our FAQs. Most responses immediately improved. ChatGPT, Google Gemini and Perplexity are all now reasonably close or correct. Google AI Overviews is still returning costs ranging from $300,000 to over $900,000 annually, citing our own website as the source. Our actual minimum is about $10,000 per month. The AI found our content and still got it wrong.

The model is not trying to mislead anyone. It is filling a vacuum. But the patient on the other end of that exchange does not know that. They are making real decisions about their health and care based on what they just read.

Why AI hallucinations are a patient safety issue

Other industries absorb AI hallucinations with a bad review and a correction. Health care absorbs them differently. A patient who receives wrong information about a procedure might delay care or even undergo incorrect treatments. One who gets a fabricated answer about a medication may act on it. One who misunderstands what a health system offers may never call at all. These same dynamics apply when patients ask about prescriptions, treatments or clinical trials. AI that fills gaps in a drug’s information with outdated, off-label or inaccurate information does not just create compliance risks for pharmaceutical companies. It further erodes already fragile public trust. This is no longer just a communications problem. In health care, information gaps can become major patient safety risks.

We’re seeing data that backs this up. A recent Nature Medicine study found that ChatGPT Health undertriaged 52% of true emergencies, directing patients with conditions like diabetic ketoacidosis and impending respiratory failure to a 24- to 48-hour evaluation rather than the emergency department. These are the exact moments when a wrong answer can cost someone their life.

This is happening against the backdrop of public trust in health care, which is already in serious trouble. According to Gallup, confidence in the U.S. health care system has fallen by nearly 49 points since its peak in 1975, with just 32% of Americans expressing a great deal or quite a lot of confidence in the health care system today. Personal physicians remain the last intact anchor of trust, but every other institutional measure is declining.

Patients are already using AI tools to navigate their care, and those tools are filling the gaps in your content with whatever they can scrape from the internet. The stakes here are not the same as getting a restaurant or hotel recommendation wrong.

Information gaps carry consequences for patients

What hasn’t fully sunk in for many health care leaders is that it used to be enough to have accurate information on your website. The standard now is whether your site has complete information, whether you have anticipated every question a patient could realistically ask and made sure your answer is findable, clear and accurate enough for an AI to interpret and correctly attribute to you.

If you have not answered the question, unfortunately, someone else has. And AI will use whatever it finds. This is the same issue that has always made silence dangerous in a crisis. The organizations that get hurt worst in a communications breakdown are often the ones that said nothing at all. That dynamic is now operating quietly at scale every time a patient turns to an AI tool before turning to a provider.

The implication is hard to ignore: Health care organizations can no longer treat digital content as simply a marketing responsibility. In the AI era, content completeness is becoming a patient safety function, not a marketing function. Health systems have compliance teams for billing, privacy and clinical quality. Few have anyone accountable for what AI is telling patients about their care.

AI hallucinations are quiet, unwanted additions to the risks and complexities of health care. The leaders who will maintain patient trust are the ones who treat closing high‑stakes information gaps as a core patient safety responsibility, not just another marketing task.

Stewart Gandolf, MBA, is CEO of Healthcare Success, one of the nation's leading health care marketing and digital strategy agencies. Since 2006, he has advised more than 1,500 health care organizations ranging from individual practices to multibillion-dollar health systems.