How physician practices can decide what to hand to AI and what to keep human

Almost 70% of consumers will give up on an artificial intelligence (AI) customer service tool after it fails them twice, according to a survey Zingtree released Oct. 6.

The survey of 1,040 U.S. adults, conducted by CMSWire INSIGHTS, asked about complex support issues: those that take multiple interactions or more than 30 minutes to resolve, or that involve a high-stakes financial or health care matter. Half of respondents said they had to repeat information to a human representative after being transferred from AI.

Booking a patient who calls with an urgent symptom into a routine appointment slot can be life-threatening, said Juan Jaysingh, CEO of Zingtree.

Signing up a minor for physical therapy without a guardian's approval on record is noncompliant, he said. In workflows like those, he said, a human needs to be involved.

Jaysingh has led Zingtree since January 2020. The Palo Alto, California, company describes itself as an AI agent governance layer for complex customer experience, software designed to keep both human staff and AI agents following an organization's own rules, procedures and compliance requirements.

Its clients in regulated industries include Allianz, Corpay, Experian and UnitedHealth Group, and its health care work includes large insurers, physical therapy clinics, eye care and behavioral health groups. Before Zingtree, Jaysingh founded ZeeMee, a community platform for high school students transitioning to college that was named to CNBC's inaugural Upstart 25 list in 2017.

He also said accountability for an AI tool's mistakes falls on the organization that chooses to deploy it, whether the tool is built in-house or bought from a vendor.

Jaysingh spoke with Medical Economics about which tasks practices can hand to AI, what front desk staff need when an AI phone agent or chatbot hands off a patient, the privacy protections to require and how to test a vendor before signing a multiyear contract.

Our conversation, lightly edited for length and clarity, follows.

Could you tell me a little about Zingtree and your own background?

Juan Jaysingh: Zingtree is an AI agent governance layer for complex patient experience. What this means is we're a unified layer for our customers, who put their knowledge, their standard operating procedures [SOPs] and their compliance requirements into Zingtree. Our solution ensures that all human agents and AI agents are following our customers' rules, logic and SOPs.

A lot of the work we do is in health care, with large insurance companies, physical therapy clinics, eye care and behavioral health groups. For all those patient experience use cases, any mistake, a wrong step in scheduling a patient for an appointment, can be critical.

If it's an urgent symptom but it's put into a routine slot, that can be life-threatening. Signing up a minor for physical therapy without a guardian's approval on record in their patient information is noncompliant. Those are the kinds of complex use cases we handle.

Personally, I can say I'm fortunate to be living the American dream. I was born in India and moved to the U.S. at the age of 14, and I was fortunate to meet the right people. I started playing tennis in India, and when I came here, I continued my passion for tennis.

I ended up getting a scholarship to play tennis in high school and in college, and tennis helped me get my first job. I got into IT consulting and worked at companies like Verizon, and then I moved into being an entrepreneur.

I started my first company about 12 years ago. I moved to the Bay Area from the D.C. metro area and started a company called ZeeMee, which focused on the college education space.

We built a community for high school students transitioning to college, and we were part of CNBC's top 25 startups, so that was fun recognition. I also did a sports tech company where I was one of the owners, and then I became the CEO of Zingtree in 2020.

Practice leaders are hearing about AI solutions all over the place, and a lot of them are turning to them for answering calls and responding to patient messages. From your perspective, how should practices decide what's OK to hand off to AI and what should remain a human's job?

You're 100% right. AI is everywhere, top of mind for all sorts of organizations, including organizations in the health care industry. To answer your question, it really starts from the top, from leadership, and then it trickles down to everyone operating at a managerial level or even an individual contributor level.

Look at how you take your rules, your logic and your compliance and use them in a way that makes life easier for, number one, your patient, and number two, the people supporting the patient, so that both user groups have a great experience.

When you look at it from that angle, what we have seen is that AI does well when you take the busy work that humans do, whether that's the end user, like the patient, or a call center agent supporting the patient. There's a lot of busy work they may do in terms of research or summarizing a call, and those things are well prepped to be handled by AI.

At the same time, there are mission-critical workflows. Like I mentioned earlier, making a mistake when a patient is calling about their symptoms, not treating it as critical when it is a critical symptom, can be life-threatening.

It's absolutely critical for a human to be involved in those kinds of workflows, because getting it wrong has a direct negative impact on revenue and a negative impact on brand recognition. So it really comes down to how you divide up these types of use cases between what can be done through AI and what can be done by a human.

At Zingtree, we like to focus on the problem first. If a patient is calling, what are they calling about? Identifying that can be done through AI. Then, how can you help them resolve that issue on that first call? Look at what pieces of that first call can be handled by AI and what pieces need to be handled by a human to make sure that patient has a great experience.

The busy work, the conversations and the summarization of those conversations can be handled by AI. But what we're really proud of at Zingtree is understanding the rules and the compliance that the organization or hospital needs to adhere to, and bringing our solutions and our systems to make sure we handle the facts accurately, 100% of the time. That allows our customers to be confident using AI at the right times, with the flexibility to make sure their rules, logic and compliance are accurate and controlled.

When AI passes a patient off to a real person, what commonly goes wrong, and what does the staff member need in front of them to prevent it?

This is the most common issue we see, and I think we've all experienced it in our personal lives. You end up talking to a bot or an AI solution, you give it all the information, and then you get handed over to a human and start all over again answering the same questions. This happens in health care as well.

Understanding the intent, why someone calls, and the context that's shared during the call is super critical, and so is transitioning that seamlessly when you get to a spot where the AI can't go any further.

Again, I'll use the example of scheduling a patient to go to the right clinic. If the AI doesn't give that accurate answer every time, you can't put it in the AI's hands. You have to put it in human hands to make sure the patient with X, Y and Z symptoms is directed to that specific clinic 100% of the time.

Think about the initial stage, before the patient is seen by the doctor, like their patient information and intake form, or after they see the doctor, like billing questions or insurance questions.

Some of those interactions can be done by AI, but there are going to be situations where you can't put it in the hands of the AI, where it has to be 100% accurate all the time. That's where you bring the human into the mix, and that transition from the AI to the human needs to be seamless.

That is one of the biggest frustrations we've seen in the research we've done. We recently got a survey back, and as excited as humans are about using AI, when they do get transferred from AI to a human, 50% of the time they find out they have to repeat the context and the answers they shared with the AI. That is definitely a hot button, and it makes it very difficult for humans to trust AI if we don't get it right.

What we like to do is what we call "human on the loop." Yes, you leverage AI, but a human is monitoring, and the minute the human needs to step in, they're able to step in. The AI is able to give them all the context it gathered from the interaction it has already had with that patient, so the human is not starting from scratch.

They monitor what's happening, and they know exactly when to interject. They make an approval or solve something that needs more detail to get accurate 100% of the time, and then they can send it back to the AI and continue to monitor.

Having that kind of seamless transition, both from AI to human and back from human to AI, is critical, and it's something we see well received by both the patient and the hospital systems and clinics supporting these patients.

Say an AI tool tells a patient something wrong, like that they can wait on something that should be treated more urgently. Who's responsible in that case? Is it the practice that implemented the AI tool, the physician who runs the practice or the company that built the tool? Where does that liability fall?

That's an interesting one. In my personal opinion, this should really fall on the providers, on the organizations that make the decision to use AI tools, whether it's a homegrown AI tool or a third-party AI tool they purchase for these specific use cases.

They're the ones making the final decision, and they understand their SOPs, their rules and the compliance they need to follow. In my view, they need to be held responsible for the experience they provide to their users, whether that's a human agent or their patients.

They need to understand the situation and make the right moves by leveraging the right tools and technology to support these types of use cases and experiences. So in my view, accountability should be held by the organizations that are implementing these AI solutions.

It's a responsibility for all of us, especially people in decision-making positions. AI is definitely super helpful. We've seen that personally in our lives, but also professionally. But it's a real responsibility for decision-makers and leaders to implement AI in a thoughtful, safe manner, where it can do its good and not run into issues that could be not just frustrating but a huge risk factor for people dealing with AI.

Health care particularly has strict privacy rules. What protections should a practice insist on before it lets AI talk to its patients?

You're right. Every health care organization takes that very seriously, so it's great to see. The standard thing we see is that you've got to make sure the AI tools you use and the vendors you work with are aligned with your organization's regulations around being HIPAA compliant and SOC 2 compliant. All those health care certifications are baseline standards, so they need to have those in place.

Second, a lot of these organizations have been around for a long time, and they've built out their SOPs and their business rules. Those need to be hard-coded into their day-to-day operations and workflows, especially the ones that need to be followed accurately 100% of the time.

The last one I would say is, once you deploy the AI solution, make sure it's working the way it should. Having the audit trail to make sure the compliance steps are taken and followed is critical.

So those three things are really important: having the right certifications to be able to get access to the right information; taking your business rules and logic and hard-coding them into your day-to-day workflows, especially the ones that need to be accurate 100% of the time; and finally, having an audit trail of how your AI solution is working, so you can see whether it's checking the box on following all the right procedures. If it's not, then you can go make those changes to make it accurate.

Some people might worry that AI will be one more wall patients have to fight through, like those automated phone menus. How do you stop that from happening and actually make it an improvement in the quality of the patient experience?

That's top of mind for all of us. AI is democratized now. You and I use AI pretty much every day, so we see the benefits of it, and we end up being both patients and operators of businesses.

So there's definitely excitement, energy and a willingness to use AI. In the research we did recently, we saw that most people want to use AI, if they can, to resolve their support questions or even complex issues. However, if the AI gets it wrong, after two failures, 70% of those people who are into AI will abandon it.

The challenge you run into there is changing behavior. We've all seen chatbots that make you run around in circles, so when you make a call now, you just press zero and try to get to a human agent. But in this new world, people are willing to give AI a chance. So it's really on the organizations and the providers to step up, take that willingness people have to use AI and deliver the right value. Because if you lose that momentum, like I said, 70% of people will abandon AI.

With that in mind, you need to be thoughtful when you implement AI. You need to understand your own organization's standard operating procedures and rules and what the right guardrails are, and that needs to translate into the systems you're using to run these AI solutions.

If you can balance where AI works well with where you need the human to step in, and that can be a seamless handoff, once you put those pieces in place and start executing, that's a great way to keep the momentum going, get buy-in from the people who are into AI and start showing them results, so they get more confidence that AI and humans can work well together.

Think about scheduling. It seems pretty simple. One of our customers was trying to have AI do the scheduling for their patients and figure out which clinic to send them to. The challenge was, when they used AI, it wasn't giving them a consistent answer 100% of the time. Maybe it got it right eight or nine times, but those one or two times can be life-threatening. So you can't take that risk when you're in the health care space.

Make sure you have the right protocols, where certain things can be handled by AI during that interaction. But when you need to understand their medical history, what type of guardians they may need for a procedure, what type of symptoms they have and what type of clinic you need to send them to, that needs to be accurate 100% of the time, and you do need a human there.

Having that blend is really critical to take advantage of this opportunity in front of you, where people are interested in AI, and to deliver value that is accurate and a great experience. Then that momentum will continue to build, more people will trust AI and that behavior change will happen faster. But if you don't get it right, you're going to be in that same circle we've all been in with chatbots, taking us for a spin, and then you start all over again. That's what we're trying to avoid here.

Can AI make staff better at their jobs, and not just pick up what the automation can't handle? What would that look like at the front desk? And should those workers be worried about their jobs?

What we've seen is that those workers are extremely happy AI is making their jobs much easier. For example, one of our customers is an orthopedic group with people at the front desk taking these calls. There's a lot of busy work they have to do before and after a call, especially after, when they have to write a summary of what took place. With the help of AI, that customer was able to reduce the after-call workload by 85% and eventually save seven minutes per call, which is a huge impact.

So yes, the people at the front desk are extremely excited about AI, because they're in the line of fire. They're taking critical calls, and they have to be on their game all the time. AI can help them prepare before the call and, after the call, put the right information in place. It's a huge advantage for those people, so they can do what they really signed up to do, which is give their experience and their empathy to that human patient who is looking for help.

Whether a physician owns a small practice or works for a big health system, what should they ask before signing on with an AI vendor? And beyond audit trails, how would they know down the line whether it's working out?

This is a really critical step, and it's a tough one, because when we talk to folks in these decision-making positions, it's not easy. They're getting 20 different companies calling them every week, and they're getting bombarded with similar messaging, so it's a really hard position they're in. With that said, they do have a responsibility to their patients, to their organization and to the advancement of AI. They're in a great spot to make a huge impact, so they need to be thoughtful about who they sign up to do business with.

There are a couple of things I would suggest. One is that it starts with the people. Who are the people you're working with? Who is going to be your partner in this engagement? Are they trustworthy? Is this a good working relationship? Is this going to work between your team and their team? As much as we're talking about AI, at the end of the day, it's people making these decisions and interacting with each other.

Then I would go one level deeper, to the people within your organization. Can they use this solution? Can they easily make updates and changes to your organization's SOPs and rules within this solution?

Also understand how they would handle the AI when it gets something wrong, when the AI is not sure about the answer or doesn't get it 100% right. How will the partner you work with handle those situations? That is critical in the complex use cases we discussed before.

The other critical thing we discussed is a seamless handoff. If there's a situation where the AI can't take it all the way to the finish line, how does that transition happen seamlessly to the human? And is there an opportunity for the human to then transition it back to the AI?

I would encourage leaders in the health care space to dive into those things. What type of people are they working with? Is there good chemistry and good trust there? Can their teams manage the solution with the knowledge they already have? When the AI isn't sure, what's the backup? And is that seamless transition from AI to human and human back to AI possible?

In terms of how you evaluate these things, yes, you can ask these questions before you sign up. But instead of signing five-year deals or three-year deals, which I'm all for, I would really encourage leaders to start with a two- to four-week POC [proof of concept]. You don't need a whole year. You can condense it into two to four weeks, or even 30 to 60 days.

Can they handle certain things that are critical for your organization? First call resolution. How do they do the transfers from tier one to tier two support, or from AI to human? What are they doing to reduce errors in scheduling patients to the right clinics? How long does it take to do the pre-call and post-call busy work, and is that time reduced with the solution? How long does it take to bring a new hire up to speed on using these tools and understanding your organization's SOPs?

There may be more that make sense for your organization, but you can pick a few of these critical things that are necessary for you to make an impact and see whether you get that type of positive result in a short POC window, two to four weeks or 30 to 60 days. From there, you have the confidence that this tool or solution has the right foundation and can deliver the value. You've seen glimpses of it, and then you can go into a more long-term relationship.

Is there anything we didn't talk about today that you think physicians should keep in mind?

It really is a great time for these physicians, these decision-makers and people running health care companies to take advantage of AI, and a lot of them are doing it. At the same time, for a lot of them, step one has been trying it. When you have something new, you want to be bold and try it, and some have unfortunately learned some hard lessons. I think that's a positive thing.

My thought is that they should continue down this path of bringing AI to the core of their business practice, but take the learnings they've had, their industry has had and their colleagues have had to heart, and be more thoughtful in how they implement AI. And keep in mind that AI is not the solution for everything.

These folks do this very well. They're doing this job because they care about the human. They care about the patient. That's the No. 1 goal. Patient experience is what they wake up every morning thinking about and go to bed every night thinking about, so they already have that in their ethos.

Bring that to the mix, and then see how to bring the right technology solution to solve for it. Some of that could be done with AI, some can be done with humans and some can be done with a mix of AI and humans.

When you look at it from that angle, it tells you what your real goals are. You focus on the patient experience, you focus on their pain points, and then you see how you bring technology into the mix, versus bringing the technology in and trying to force it into a specific use case. That tends not to go well a lot of times.

So I would leave by saying that it's a great time.

The folks in the health care space are really set up to be very successful implementing AI, because at their core, they care about the people and they care about the patients.

It's going to be an exciting time over the next five to 10 years, with how AI is going to be implemented in the industry, and we're excited at Zingtree to be a small part of that.