AI chatbot treatment recommendations are swayed by drug ads

A clearly labeled pharmaceutical advertisement placed next to a clinical question can shift which treatment an artificial intelligence (AI) model recommends, according to a study from researchers at the Icahn School of Medicine at Mount Sinai published Nov. 11 in The Lancet Digital Health.

The researchers tested 12 AI models from major technology companies, including Google, OpenAI and Anthropic. Across 33 clinical scenarios, the models chose the advertised drug in 47.6% of responses when an ad was present, compared with 35% when no ad was shown. In four cases, models went from never choosing the advertised drug to choosing it 100% of the time once the ad was added.

The findings come as more patients turn to AI for health information, and they raise questions about how commercial content could shape the answers those patients and their physicians receive.

The study was designed to isolate the effect of advertising. Researchers gave the models the same physician-written clinical cases, either alone or with a real, clearly labeled drug advertisement placed immediately before the case. The patient information and treatment choices stayed the same; only the presence of the ad changed. In all, the team analyzed 258,559 AI responses, assessing both which treatment each model selected and whether that choice was consistent with clinical guidance.

According to the researchers, the shift often happened among treatment options that were all clinically appropriate, which can make the influence hard to spot.

“What I find quite interesting, and even surprising, was how reasonable the answers often appeared,” said first author Mahmud Omar, MD, a physician-scientist and lecturer in the Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine. “The model could recommend an appropriate treatment, but its preference among several reasonable options changed after seeing the ad.”

Omar, who leads research on the safety, reliability and real-world effects of generative AI in clinical care, said that because the final answer could still look medically sound, “that influence could be easy to miss in typical research evaluations” and in personal use by patients and physicians.

The models also rarely acknowledged the ad. In a subset of free-text responses, models explicitly mentioned the advertisement in only 29.9% of responses, though they repeated information or claims from the ad more often, the researchers reported.

“An answer can be clinically reasonable and still raise an important question about why the AI arrived at that answer,” said senior author Girish N. Nadkarni, MD, MPH, chair of the Windreich Department and chief AI officer of the Mount Sinai Health System. “As AI becomes part of how people get health information, we need to be able to distinguish clinical guidance from commercial influence and be transparent about when advertising is part of the interaction.”

The investigators cautioned that the results do not mean drug advertising will necessarily lead AI models to recommend unsafe or inappropriate treatments. In the study, ads did not clearly increase recommendations for unsupported supplements or clearly suboptimal drugs. The researchers also did not examine whether changes in AI recommendations affected prescriptions, patient decisions, costs or health outcomes.

The researchers said the findings point to several practical steps for AI developers and health systems: testing AI systems with and without promotional content to see whether recommendations change, clearly disclosing when advertising is presented to an AI system and keeping promotional material separate from clinical advice.

The study may also matter to regulators as they weigh how existing rules governing pharmaceutical advertising apply when promotional content becomes part of AI-generated health information.

The Mount Sinai team plans to extend the work through its ongoing evaluation of AI and human-AI interaction in real-world clinical cases.

The paper, “Ad-verse Effects: Pharmaceutical Advertising and Drug Recommendations by Consumer-Facing AI in a Controlled Study,” was supported in part by grants from the National Center for Advancing Translational Sciences and the National Institutes of Health.