What are the data security risks of using AI tools in healthcare?

Healthcare organizations have spent decades building infrastructure to protect patient data — but generative AI has expanded the attack surface faster than most compliance programs can keep up. A recent industry report found that the majority of data policy violations in healthcare now involve regulated information, much of it flowing through consumer AI tools employees weren't authorized to use.

Harley Sugarman, founder and CEO of Anagram Security, has spent his career helping organizations close that gap — not by writing longer policies, but by training employees to recognize the security risks hiding inside their everyday habits. His work focuses on healthcare specifically, where staff manage some of the most sensitive data that exists, often under intense time pressure, with little bandwidth to stop and evaluate which AI tool is safe to use for a given task.

As AI becomes embedded directly into clinical workflows — from transcription tools to platforms like Microsoft Copilot integrated into Epic — the line between sanctioned and unsanctioned AI use is getting harder to see. Physician practices are being asked to weigh the productivity gains of these tools against real HIPAA exposure and vendor risk, which is part of why building a practical AI compliance framework has become such a pressing conversation.

Medical Economics spoke with Sugarman about this issue to learn more.

(Editor's note: The following transcript has been edited for brevity and clarity.)

Medical Economics: When you look at how clinical and administrative staff in a healthcare setting are actually using AI tools, what are some of the most common data exposure behaviors you might see?

Harley Sugarman: Healthcare workers deal with the highest level of sensitive data day to day — it's not just an email address, or even credit card information. It's someone's deepest, most sensitive information: their healthcare records. Up to this point, there's been fairly sophisticated infrastructure and tooling in place for healthcare workers to manage the risk of that data being exposed, weighing efficiency and benefits against the security risks of having that data live on the internet.

What's happened with AI is that it's expanded the surface area of where this data can live. It used to be very clean — you knew this data was living in your database or your EHR, and when you transmitted it, you had standardized protocols like encrypted email or encrypted file transfers to get it from point A to point B. Now AI exists within a lot of that software, and I think of two buckets when it comes to how AI can be used safely, or less safely, in this space. The safe bucket is when AI lives inside the tools healthcare workers are already using. In those cases, it's easy to validate that the vendor has gone through security checks to make sure data is being stored according to HIPAA standards.

Where it gets more confusing is when you get into tools outside of AI, outside of the medical space — things like transcription, file storage, and processing of charting, which are genuinely powerful for medical professionals. But now you're not just looking at the company providing that note-taking software and asking what databases they store data on. You have to look at what subprocessors they're using, what language models, whether they're using public models or a hosted model, and where that data is being hosted. Suddenly you're looking at three to four orders more potential vendors and links in the chain, because you don't have the control you had with more traditional SaaS.

Medical Economics: Somebody's using ChatGPT, for example, to solve a problem, and they put in some patient information they shouldn't have out in public. What's the worst thing that could happen?

Sugarman: The theoretical worst case is: I'm a doctor, and I type a patient's symptoms into my personal ChatGPT, which isn't signed in and is on the free plan. Any data I give ChatGPT then belongs to OpenAI, and they can do what they want with it. Hypothetically, that data could get exposed to somebody else because it goes into ChatGPT's training data, and could then show up in ChatGPT's output to other people. That's the theoretical worst case, and I think it's very unlikely to happen.

However, there are a lot of legal guardrails around sharing sensitive personal information — specifically personal health information — with public tools. If you share something with a system that doesn't have protections in place, or that you have no guarantee about what's being done with the data, you've immediately committed a HIPAA violation. There's also legal risk around age and consent, since there's no guarantee the patient's information is coming from someone over 18. There's a lot of legal risk the second you put anything into a language model. In terms of whether that worst case actually happens — where a patient's personal information shows up in somebody else's ecosystem — it's unlikely, but it's theoretically possible.

Medical Economics: You work with training employees to recognize security risks. In healthcare, staff are often overwhelmed. What makes it so hard for people to pause and think about these security issues before pasting something into an AI tool?

Sugarman: You have to put into perspective what medical workers are dealing with day to day — it's legitimately life and death in a lot of cases. Oftentimes the last thing on their mind is whether they're adhering to a specific legal framework, or whether they've signed into the right account on this machine, or whether they're using the right tool for their company. They just want to save the patient's life or make the patient healthy again.

So as the orchestrator of the tools employees are using, you have to understand that and build processes, systems, and guardrails that make it easy for them to do the right thing, even given the constraints. It's an unreasonable ask for a healthcare worker under an incredibly high amount of stress to stop and think about whether they're using the right tool, or which tab they should paste into. Humans are humans, and we make mistakes. What's really important is that IT professionals and security teams set up the guardrails so the defaults are sensible and make the right choice easy.

Medical Economics: Do you find that the exposure risks are coming from front-line, lower-level employees, or is it happening throughout the organization, all the way up into the C-suite?

Sugarman: It's throughout — though the level of exposure and risk is different as you move through the different levels of the organization. At the more boots-on-the-ground level, you'll get very direct, first-order information — maybe listening to a patient and transcribing what they said, or transcribing real records that were just delivered. There's risk there, and there are a lot of constraints around how you interact with that data, but typically that's in the single-to-double digits in terms of number of records touched. It's very rare that a single medical practitioner is touching tens of thousands of medical records.

As you go up in the organization, you start getting less direct personal information, and there are usually more guardrails in place to obfuscate and anonymize the data and make it harder to access. But if you do get breached at that level — if those guardrails get overridden by someone with privileged access — the downstream effect can be a lot worse, because the number of records they have access to is potentially much greater.

Medical Economics: A report from Verizon found that two-thirds of employees are accessing AI through personal, unapproved accounts. Healthcare leaders often assume that because they have a policy prohibiting that, they don't have to worry about it. Why is that a risk — just assuming that because you have a policy, people are going to follow it?

Sugarman: It's very easy to put a policy in place and assume people will listen to it, because you get the legal comfort of having said they can't do this. The problem is that policy is inherently complicated — a lot of hands go into writing it, often legal hands that don't necessarily speak in plain English. So you have two issues: first, do people actually understand what the policy says? And second, how do you know people have heard it, understood it, and internalized it?

You can solve that in different ways. AI is actually a great way to solve the first problem — it can take a piece of policy and translate it into something digestible and relevant to a specific healthcare worker. It's really useful for a healthcare worker to look at a policy and ask, "How does this apply to me as a nurse practitioner, or as a surgeon, or as a dentist?" AI is very good at walking you through that and dissolving ambiguity.

Where it gets tricky — and this was true before AI even existed — is making sure people actually adhere to those policies once they've read and understood them. That's where it comes back to having smart guardrails and good training in place. One thing we believe strongly is that a lot of existing corporate training is pretty dreadful — 45 minutes of videos that sound like they were written by lawyers, followed by reading-comprehension quizzes that feel like the SAT instead of actual learning.

Our philosophy is that you have to ask who you're training, how you're training them, and why it's important. For healthcare workers, training that puts them in the shoes of a marketing employee or an engineer or someone at a bank doesn't apply — their world is too different. You have to make it relevant, show that you understand the ins and outs of their world, and operate within the constraints they deal with. Most regular employees don't have the attention span for 45 minutes of training as it is; now imagine if you're seeing 20 patients, or working in an emergency room. You're never going to get the engagement you need to actually internalize it.

Medical Economics: As AI gets embedded more directly into electronic health records or clinical decision support tools — for instance, Microsoft Copilot is in many versions of Epic — does that reduce the shadow AI problem, or does it create new risks that healthcare organizations aren't prepared for?

Sugarman: It reduces the shadow AI problem if people actually use it. If your hospital is a Microsoft shop, you're all on Outlook, you have your Epic subscription, and Copilot is integrated into both nicely, that's a great place for your IT team to be — they have agreements with Epic and Microsoft, and reasonably high confidence that those companies know what they're doing when it comes to cybersecurity.

Where it gets dangerous is that those built-in tools aren't always the best or most effective tools for the job — you see this outside of the medical industry too. What's great about AI, and also what's scary about it, is that it's so easy: you can sign up for a tool and, out of the box with no configuration, it can do a task to a level you wouldn't have thought possible three years ago. If the Microsoft or Epic version of that tool is only 20% as good, it's going to be really difficult for an employee not to take the shortcut, because it makes them better and more effective at their job.

The stakes are just higher here, because you're dealing with very sensitive data and often fairly subjective decisions that a medical practitioner has to be careful about and liable for. If you start outsourcing those decisions — whether it's Copilot inside Epic or outside it — you have to be very careful. Although my suspicion is that if you're using Epic or Microsoft, they've probably built in a lot of guardrails to avoid that kind of exposure.

Medical Economics: If a physician group practice could only do one thing right now to reduce their AI-related data exposure risk, what would you tell them to do first?

Sugarman: I'd tell them to audit which tools their employees actually like and find most useful, then build good vendor relationships with those products if they can. Just because you're paying for Microsoft and Copilot is bundled in doesn't mean Copilot is the best tool for what you need. It's really important for security teams to understand the business case for different AI tools — why they're being used, why people want to use them, why they don't want to use the one you've already bought for them.

Once you have that understanding, you can make the business case: "It looks like our practitioners much prefer the transcription capability of Claude because it's more accurate, with fewer false positives, and it saves us 50% of the time we spent on task A, whereas Copilot saves us maybe 10%, but there are so many mistakes we still need to read everything in triplicate." Once you've done that, you can have the conversation with that company and say, "We need you to give us this level of assurance around what the data is going to do, where it's going to live, and which compliance frameworks we adhere to."

That enables users to use AI in the way they want to use it. You won't get 100% coverage, and some people will be unhappy — that's always going to be true. But it's important to try to empower users to work the way they want to work, because that's what makes them the most effective and productive, and it's how you get ROI from these AI tools more quickly, rather than giving people tools they have to fight with.

Medical Economics: Any other thoughts for physicians running medical practices that we haven't talked about?

Sugarman: In the medical space there's so much urgency, and there's a lot I — someone who works behind a desk at a computer all day — don't fully appreciate. There's a lot of subtlety in how these users work and how they'll work with AI, and it looks very different from how someone on a security team thinks about AI. It really comes down to empathy: building relationships with the business units and the practitioners themselves to understand how this stuff is actually being used, and then effectively building guardrails that make it easy for them to do their job.