Microsoft says its diagnostic AI is now peer reviewed, and the headline number is hard to ignore. In a paper published in Science on October 22, the company’s Diagnostic Orchestrator, branded MAI-DxO, correctly diagnosed 83.2% of notoriously difficult medical cases. Generalist physicians given the same cases managed 22.3% working from memory and 36.1% when they could look things up.
Microsoft AI CEO Mustafa Suleyman has been promoting the result hard, and as UNILAD Tech reports, the framing that has stuck is that the AI is “four times more accurate than a human doctor.” The study is real and the jump is large. But “four times” only works for one comparison, the cost savings depend on settings, and nobody has used this thing on a live patient yet. Here is the full picture.
The short version
- The paper, “Sequential diagnosis with language models,” appeared in Science on October 22, 2026. All 15 authors work at Microsoft.
- On 304 hard New England Journal of Medicine case records, MAI-DxO running on GPT-5 scored 83.2%. Generalist doctors scored 22.3% without references and 36.1% with them.
- On real emergency department admissions it scored 80.3% versus 46.8% for physicians, about 1.7 times better.
- “Four times” is really 3.7 times, and only against doctors who could not use books or the internet.
- The first live test, at Beth Israel Deaconess Medical Center in Boston, should start “in the next couple of months.”
How the test actually works
Most AI medical benchmarks hand a model the full case and ask for a diagnosis, which is a bit like giving a student the answer sheet. Microsoft built something closer to a real workup, called the Sequential Diagnosis Benchmark, or SDBench. The diagnostician, human or AI, starts with only a short summary of the patient. From there it has to ask questions and order tests one at a time before committing to an answer.
A separate AI “gatekeeper” plays the role of the patient’s chart, revealing findings only when they are specifically asked for. Every test has a price attached, estimated from real hospital billing codes, so the benchmark measures both accuracy and how much money was spent getting there. An AI “judge” then grades the final diagnosis against the published answer using a rubric written by physicians.
The cases came from two places. The first set is 304 clinicopathological conference cases from the New England Journal of Medicine, the famously tricky diagnostic puzzles doctors read for fun and for humility. The second, new in the Science version, is 325 emergency department to hospital admission records from MIMIC-IV-ED, a research dataset drawn from Beth Israel Deaconess in Boston.
MAI-DxO itself is not a new model. It is an “orchestrator” that makes a large language model behave like a panel of doctors, with separate roles proposing hypotheses, choosing tests, challenging assumptions and watching the budget. In the paper it runs on OpenAI’s GPT-5, and Microsoft says it reran the whole suite with GPT-5.5 before publication.
Where “four times more accurate” comes from
The quick math: 83.2% divided by 22.3% is about 3.7. That is the comparison against generalist physicians who were not allowed to use textbooks, search engines or colleagues, a rule the team set because NEJM cases are published online and could simply be looked up. When the doctors could use references, the gap shrinks to about 2.3 times. On the emergency department cases, which look a lot more like ordinary medicine, Microsoft itself describes the gain as 1.7 times.
The authors are upfront about the limits. Their own paper says the results do not show the AI is better than specialists, multidisciplinary teams or real clinical practice, and that they would expect the gap to narrow against specialists. Complex NEJM cases are usually handled by exactly those people, not by a lone generalist working through a text interface. The setup was also text only, with no imaging, pathology slides or physical exam, and the judge scored the diagnosis alone, not treatment or safety.
The cost claim depends on a setting
Microsoft has stressed that MAI-DxO reaches better answers while spending less on tests. That is true in the budget-limited configurations. In “budget” mode on the NEJM cases it hit 80.9% while spending an estimated $2,241 per case, less than either physician group. In the unrestricted mode behind the 83.2% headline, though, it spent about $5,431 per case, more than the $4,725 spent by physicians with references. On emergency cases the no-budget version spent $4,979 against $3,400 for physicians.
There is a second, quieter caveat. A lot of the improvement comes from the underlying model, not the orchestration layer. TechCrunch notes that plain GPT-5 with a basic prompt already scored 78.9% on the NEJM cases. MAI-DxO added about four points and cut roughly $3,000 per case, while on the emergency cases it delivered “little improvement in either accuracy or cost” over GPT-5 alone. Where the orchestrator shines is on weaker models: Gemini 2.5 Flash jumped from 46.4% to about 69% with it.
| 2025 preprint | 2026 Science paper | |
|---|---|---|
| Base model | OpenAI o3 | GPT-5, rerun on GPT-5.5 |
| Top AI accuracy | 85.5% (max-accuracy mode) | 83.2% (no budget mode) |
| Physician accuracy | About 20%, 21 doctors, no references | 22.3% without, 36.1% with references |
| Case sets | NEJM cases only | NEJM plus 325 ED admissions |
| Peer reviewed | No | Yes |
Next stop: a Boston emergency room
Suleyman is careful on one point. “It needs clinical validation,” he told The Verge. “That is a leap, but this is a major milestone along that journey.”
That step is planned at Beth Israel Deaconess Medical Center. Dr. Dominic King, who leads health at Microsoft AI, said the study is likely to “get underway in the next couple of months,” with no firm date. According to TechCrunch, patients will get normal care from their doctors, and those who consent will also interact with MAI-DxO in parallel so researchers can compare its recommendations with what the clinicians actually decided. It will cover emergency and urgent care settings. “We think this will be the biggest study of its kind,” King said.
Microsoft has not said when, or whether, MAI-DxO becomes a product. We found no FDA submission for it, and King pointed to “high regulatory and compliance requirements.” The company already sells into hospitals through Dragon Copilot, its clinical documentation assistant, and launched the consumer Copilot Health app earlier this year.
Google is running the same race, in the same hospital
Microsoft is not alone. Google’s AMIE system, published in Nature in 2025, beat unassisted clinicians on NEJM case reports too. More interesting, Google has already run a prospective feasibility study with 100 real patients at, of all places, Beth Israel Deaconess, with a physician supervising live. It recorded no safety stops, but its management plans scored worse than primary care doctors’ on practicality and cost. An earlier randomized trial in JAMA Network Open in 2024 found that simply handing doctors GPT-4 did not significantly improve their diagnoses, even though the model alone did well.
That last result is the one hospitals should remember. A strong benchmark score does not automatically become better care. How the tool fits into a doctor’s workflow, who checks it, and who is liable when it is wrong matter as much as accuracy. Regulators are already wrestling with this at the edges, as when Utah let an AI write prescriptions with a doctor still signing off on the first hundred.
The “medical superintelligence” pitch
Suleyman frames all of this as a path toward what he calls medical superintelligence. He told The Verge he expects “something that is indistinguishable from human performance on medical superintelligence in the next couple of years,” and that “next year we’re going to see extremely impressive deployments of intelligent systems in production.” He has also said human clinicians will keep a central role in patient care. It is a softer tone than you might expect from someone who has previously warned about AI that only seems conscious.
The honest read: this is a genuinely strong result on a harder, more realistic benchmark than most, and the jump from preprint to peer-reviewed paper matters. It is also a company study of its own product, on historical cases, against generalists working alone. The Boston trial is where the claim gets tested for real. King himself pointed to the complexity of “a Boston emergency department on a Friday evening.” If MAI-DxO holds up there, the four times headline will matter far less than the outcome data.
Sources and further reading
- UNILAD Tech: Microsoft champions AI diagnosis tool four times more accurate than a doctor
- Science: Sequential diagnosis with language models (Nori et al., 2026)
- The Verge: Suleyman on the Diagnostic Orchestrator and medical superintelligence
- TechCrunch: Suleyman says Microsoft’s AI outperforms doctors on hard cases
- Medical Economics: AI system outperforms physicians in Microsoft study
- arXiv: 2025 preprint, Sequential Diagnosis with Language Models
- GitHub: SDBench
- Nature: Towards accurate differential diagnosis with large language models (Google AMIE)
- JAMA Network Open: LLM influence on diagnostic reasoning (2024)
About this article: GeekBlog covers U.S. technology news, AI, phones, smartwatches and gaming. Every story is written and checked under our Editorial Policy. Spotted a mistake or have a story tip? Contact our editors.
