AI scribes: 4 habits that can help protect you from liability

(Note: This is an excerpt from an episode of Vital Signs With Dr. Sarah Matt. The transcript has been edited for brevity and clarity.)

A physician uses an AI [artificial intelligence] documentation tool, and it generates a note. The physician skims it, clicks approve, and moves to the next patient. Two years later, that note is Exhibit A in a malpractice case, and the tool had written a detail that was never discussed. The chart says something happened in that visit, and it didn't. And because the physician signed off on it, it became the physician's statement. So the tool's gone. The company may not exist, but the physician is still in the case.

AI tools fail differently than people fail. There are four failure classes worth knowing. First is hallucination. Large language models generate plausible text, and in a documentation tool, that means the software can produce a note that reads like a complete and accurate visit summary, and it contains details that maybe weren't discussed. The output looks correct, and that's the problem. Errors that look wrong get caught. Errors that look right get signed.

The next is context collapse. Most of these tools pull from prior notes and the problem list to build context. A diagnosis that should have come off the list two years ago now propagates into every generated summary as though it were active, and it gets even more confident-looking each time it repeats.

Third is the confidence without calibration. A suggestion that the model is barely committed to looks identical on screen to one that it's nearly certain of.

Last is boundary drift. A tool gets validated for one population and deployed to another, and it gets built around one EHR’s [electronic health record] data structures and installed in a system with very different coding conventions. Performance degrades quickly in that scenario.

None of these is malicious, but all of them are consequential. This is not an argument against these tools. It's actually an argument for using them with a working knowledge of how they fail. How do we deal with that?

First, say it out loud and write it down. Before the software listens, the patient should hear a plain sentence. Review everything before it goes into your chart, and then document what you said.

The next is actually reading before you sign. The single largest risk in AI documentation is the physician or nurse approving a stack of notes at the end of a 30-patient day. Every generated note needs a full read before attestation.

Remember to document your reasoning, not just the output. There’s an entire difference between a record actually showing physician judgment and a record showing physician compliance.

Ask what happens to the audio. This is a question nobody really talks about. That recording is protected health information, and it's not a byproduct of a note. It's the record being on its own right, sitting on someone else's server.

Tell your patients, read your notes, show your reasoning, and know where the audio actually goes. Those four things cost you almost nothing, and they are what will separate the physicians and nurses who navigate the next five years from the physicians and nurses who spend them explaining a note they didn't write.