Microsoft says its AI tool outperforms physicians on complex diagnostic challenges

Microsoft says AI tool outperforms doctors: ©dvoevnore - stock.adobe.com

Microsoft has unveiled research showing that its artificial intelligence tool, the Microsoft AI Diagnostic Orchestrator (MAI-DxO), significantly outperforms experienced physicians in solving complex diagnostic cases, while also delivering lower associated testing costs.

In a head-to-head evaluation using case records from the New England Journal of Medicine, MAI-DxO correctly diagnosed 85.5% of the 304 cases—a rate more than four times higher than a group of 21 U.S. and U.K. physicians, who achieved 20% accuracy on average, according to the company. The AI tool also used fewer diagnostic resources than human clinicians or individual AI models tested.

“MAI-DxO represents a new frontier in AI-driven clinical reasoning,” the Microsoft AI team said in a blog post. “By orchestrating the strengths of multiple large language models, we can emulate a virtual panel of physicians that collaborate to reach accurate diagnoses with efficiency and transparency.”

Unlike standard AI benchmarks that rely on multiple-choice questions, Microsoft’s team focused on a sequential diagnostic process modeled after real-world decision making. The tool simulated how a clinician would ask follow-up questions, order tests, and refine their diagnosis step-by-step. Each diagnostic decision incurred a virtual cost, allowing the researchers to measure both diagnostic accuracy and economic efficiency.

Physicians tested in the study were limited to individual work without access to textbooks, peers, or AI tools to ensure a direct performance comparison.

The orchestrator, which integrates models such as GPT, Claude, and Gemini, also offers configurable cost constraints, allowing it to balance diagnostic thoroughness with cost-effective care—an increasingly important metric as U.S. health spending nears 20% of GDP.

AI may offer clinical decision support

While generative AI tools have demonstrated strong performance on medical licensing exams in recent years, Microsoft’s research suggests they may soon have a much deeper role in everyday clinical decision-making—especially for diagnostically complex cases.

The MAI-DxO platform is built on the idea that AI doesn’t need to choose between the depth of a specialist and the breadth of a generalist—it can have both. This flexibility allows it to match or exceed human performance in cases that span multiple organ systems, specialties, and disciplines.

Experts say AI tools may eventually provide real-time diagnostic support, reduce costly and unnecessary testing, and extend the diagnostic reach of frontline providers.

Microsoft emphasized that the tool is not intended to replace physicians, but to augment their capabilities. AI systems could help clinicians navigate rare or unfamiliar cases more confidently, while also supporting patient self-management for routine conditions.

Still, Microsoft acknowledged limitations. The research evaluated performance on complex NEJM cases, not the more routine or ambiguous cases often encountered in primary care. Also, the physicians in the study worked alone, without their usual access to resources or team collaboration.

Next steps include testing MAI-DxO in real-world clinical settings and establishing regulatory pathways for deployment. Microsoft said it is working with health organizations to evaluate safety, reliability, and practical integration.

“Important challenges remain,” the Microsoft AI team noted. “But we’re energized by the opportunities ahead. We strongly believe the future of healthcare will be shaped by augmenting human expertise and empathy with the power of machine intelligence.”