A model does not get smarter overnight just because someone wrapped new code around it. At least, that used to be the safe assumption. Last week Nvidia published results showing its AVO agent system scored a perfect 100.00 on ARC-AGI-3, the hardest interactive reasoning benchmark currently in circulation. The underlying model, Anthropic’s Claude Opus 5, scores 30.2% when it runs on its own. Nothing about the model changed between those two numbers. Only the software around it did.
That gap, 30% versus 100%, is the whole story here, and it says something uncomfortable for anyone tracking AI progress by model release alone. Frontier labs keep shipping bigger, more agentic models, but Nvidia’s result suggests a meaningful chunk of what looks like “intelligence” in these systems is actually engineering: memory, retries, and a supervisor that knows when to intervene. Here is what actually happened, and why the caveats attached to it matter almost as much as the headline number.
The Benchmark That Was Built to Resist Shortcuts
ARC-AGI-3 exists because its predecessors got solved in ways their creators did not intend. The ARC Prize Foundation, co-founded by researcher François Chollet, designed this third version to test what it calls fluid intelligence: the ability to figure out an unfamiliar system through interaction rather than recall something learned during training. There are no written rules, no stated objective, and no example solutions. An agent is dropped into a handcrafted, game-like environment and has to work out what it is even trying to do before it can try to do it.
That design choice is deliberate and it is brutal. When ARC-AGI-3 opened to the public earlier this year, frontier models running without any special scaffolding clustered under 1%. Gemini 3.1 Pro managed 0.37%. Claude Opus 4.6 scored 0.2%. The best purpose-built agent in the preview phase, with scaffolding added, reached 12.58%. Humans, for comparison, solve close to 100% of the same environments, usually within a few minutes of trial and error per level.
Progress since launch has been fast by any normal measure, and AVO’s result is really the tail end of a several-month climb rather than an isolated stunt. Getting a full run through that history helps explain why Nvidia’s number looks so different from everything that came before it.
The Numbers, Side by Side
Put next to each other, the recent scores tell two different stories depending on whether you are looking at the model or the system running it. OpenAI’s GPT-5.6 Sol, for instance, roughly triples its own score just by changing how it retains and compacts reasoning between steps, without any new model weights involved.
| System | ARC-AGI-3 score | What changed |
|---|---|---|
| Claude Opus 5, bare model | 30.2% | Current leader among models with no added scaffolding |
| GPT-5.6 Sol, official harness | 13.3% | Baseline agent setup, no memory optimizations |
| GPT-5.6 Sol, retained reasoning | 38.3% | Reasoning kept and compacted across steps instead of discarded |
| VISTA + Claude Opus 5 | High, not published as 100 | 7,542 environment actions to clear the public set |
| Nvidia AVO + Claude Opus 5 | 100.00 | Same public set, roughly 12% fewer actions than VISTA |
Read down that “what changed” column and a pattern jumps out. Nobody retrained a model to get these gains. Every improvement came from how the agent remembers, retries, and manages its own reasoning between one action and the next.
What AVO Actually Does
AVO stands for Agentic Variation Operators, and it was not originally built for game-like puzzles at all. Nvidia developed it for autonomous software engineering and GPU-kernel optimization work, the kind of long, exploratory task where an agent tries something, checks whether it worked, and adjusts, over and over, sometimes for hundreds of steps. ARC-AGI-3 turned out to reward exactly that skill set, which is part of why Nvidia entered it in the first place.
The architecture rests on two ideas. The first is persistent memory: AVO carries forward what it already tried in an environment and what happened as a result, so it stops wasting actions repeating failed exploration once a level has been partly mapped out. The second is a supervisor layer that watches the broader trajectory of an attempt, not just the next single action, and nudges the main agent when progress stalls or when it looks like it is circling the same dead end. Neither piece touches the weights of the model doing the actual reasoning. The same logic applies to smaller open-weight agent models: a lighter model paired with a well-built harness can close a surprising amount of ground against a bigger one running alone.
- Persistent memory of prior attempts and outcomes within an environment, so exploration is not repeated
- A supervisor layer that tracks the whole trajectory and intervenes when the agent stalls
- No fine-tuning or weight changes to Claude Opus 5 itself
- Roots in autonomous software engineering and GPU-kernel tuning, not game benchmarks
Read the Fine Print Before You Get Too Excited
Nvidia’s own write-up is more careful than the “100%” headline suggests, and it is worth taking that caution seriously. The company is explicit that this is not a controlled ablation study. AVO and VISTA differ in their agent backend, how they represent what the agent observes, their memory systems, and how they manage context, on top of the harness logic itself. That means you cannot cleanly credit the 12% action reduction, or the jump to a perfect score, to any single design choice. Several things changed at once.
There is also a narrower question worth asking about any benchmark that gets driven from single digits to triple digits in months: is the system getting smarter, or is it getting better at this specific test? ARC-AGI-3’s public set is exactly that, public, which means a determined team can iterate against known environments in a way that would not generalize to genuinely novel ones. The ARC Prize Foundation maintains a separate, unpublished set for that reason, and how AVO performs there has not been reported. A perfect score on the public leaderboard is real progress. It is not automatically proof that the underlying approach would handle a puzzle nobody involved has ever seen.
What This Means If You’re Building AI Agents
Most readers are not chasing benchmark leaderboards, but the underlying lesson travels well beyond ARC-AGI-3.
- The harness is not a minor implementation detail anymore. A 70-point swing on the same model is a bigger jump than most model-to-model upgrades deliver. If your team is building on top of an API model, the orchestration layer around it may matter as much as which model you pick.
- Persistent memory pays for itself on long, exploratory tasks. Anything that resembles trial and error, debugging, tool use, multi-step research, benefits from an agent that remembers what it already ruled out instead of re-discovering it every attempt.
- A supervisor layer is a cheap insurance policy against stalling. Watching the trajectory, not just the next step, catches loops and dead ends earlier than most simple agent loops do on their own.
- Don’t assume open, smaller models are automatically outclassed. A capable harness narrows the gap between a lean model and a much larger one, which is worth remembering before defaulting to the biggest model available for every task.
The Bottom Line
The most interesting number in this story is not 100. It’s the 70-point gap between what Claude Opus 5 can do alone and what it can do inside AVO. For years, the AI industry’s easiest headline was a bigger model with a bigger score. Nvidia’s result is a reminder that the software wrapped around a model, how it remembers, retries, and knows when to change course, is becoming its own competitive battleground, one that does not require a single new parameter to move the needle.
