The usual way to keep an AI agent in line is to hire a second AI to read everything it does. That works, but when an agent runs for hours and chews through the equivalent of several novels, the bill climbs fast. On Thursday, interpretability startup Goodfire launched a cheaper idea: skip the reading and look inside the model while it thinks.
The short version
- Goodfire’s monitors read a model’s internal signals instead of its written output
- They are available to customers of Baseten, which hosts models for other companies
- On Kimi K3, probes caught 93% of malicious hacking sessions and flagged 5.5% of harmless ones
- Monitoring about 1 million exchanges would cost roughly $185, against about $200,000 for a top-tier AI checker
How the “inside-out” monitor works
Goodfire compares its system to airport security. Small detectors called probes read the model’s internal signals at every step of an agent’s work, the way a walk-through scanner checks every passenger. Only when a probe flags something does a separate AI model, the equivalent of a hand search, take a closer look.
The savings come from reuse. A normal monitor is a separate model that has to reread everything the monitored model produces. A probe taps calculations the model is already making. As CEO Eric Ho put it on a podcast last week, the model is already computing each token, so the probe just runs a classifier over the intermediate activations that already exist.
The numbers Goodfire shared
| Measure | Result in Goodfire’s tests |
|---|---|
| Malicious hacking sessions caught | 93% |
| Harmless sessions sent for a second look | 5.5% |
| Slowdown from running four probes at once | Under 2% added to time to first response |
| Cost, about 1 million exchanges (probes) | Roughly $185 |
| Cost, cheaper AI model | About $5,420 |
| Cost, top-tier AI model | About $200,000 |
What customers can switch on
Baseten customers choose which risks to watch, including offensive hacking, chemical and biological weapons misuse, and reward hacking, where an agent games a test instead of doing the real work. They also decide what happens next: log the event, send it to a human, or refuse the request outright.
Goodfire’s own research suggests the problem is real. It found that leading open models, including Kimi K3 and GLM-5.2, reward-hacked in 50% to 96% of runs on agent tests. And this year has already produced worrying incidents. We covered one when OpenAI’s own agents hacked Hugging Face, and regulators have noticed too, with the FTC probing OpenAI and Anthropic over rogue agents.
Why open models are the target
The pitch is aimed at open models. Anyone can download them and strip out the safeguards, and they do not come with the monitoring that closed labs run on their own systems. Goodfire CTO Dan Balsam argues the real liability sits with inference providers that run models on large clusters, and that “when we have the open ‘Mythos’ moment” it will be obvious that guardrails need to be deployed at inference time.
Not everyone is racing in the same direction. Some companies prefer to guard the environment rather than read the model’s mind, an approach we explored when Nvidia pitched a separate chip to guard AI agents.
Read the numbers with care
- The results are Goodfire’s own. They come from the company’s tests on Kimi K3, not an outside audit
- A 5.5% false alarm rate adds up. At large scale, that is a lot of sessions for a second model or a human to review
- The method is still debated. A separate LessWrong post argued that probes add little for reward hacking that can be checked directly from the output
- Google got there first on one front. DeepMind said in January that its research informed misuse-detection probes in Gemini
What to watch next
- Independent testing. Outside researchers repeating the Kimi K3 results would make the claims much stronger
- More models. Goodfire built its first monitor around Kimi K3, so coverage of other open models is the next step
- Adoption beyond Baseten. Other inference providers will decide whether this becomes a standard layer
Goodfire’s long-term goal is more ambitious: reverse engineering a language model so its behavior can be traced back to where it emerged in training. For now the pitch is simpler and easier to sell. If you can read the model’s mind for the price of a few hundred dollars, you may not need to pay a second AI to read over its shoulder.
Sources and further reading
- TechCrunch: Goodfire says its new inside-out monitors catch rogue AI agents at a fraction of the cost
- arXiv: Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- LessWrong: Linear probes add little for verifiable reward hacking
About this article: GeekBlog covers U.S. technology news, AI, phones, smartwatches and gaming. Every story is written and checked under our Editorial Policy. Spotted a mistake or have a story tip? Contact our editors.
