Goodfire Launches Inside-Out Monitors to Catch Rogue AI Agents
Interpretability startup Goodfire has introduced a lower-cost approach to supervising AI agents by inspecting internal model signals as they operate rather than employing a separate secondary model.
Radwan AbdulhadiEditor and technology writer
A New Approach to Monitoring AI Agents
The traditional method to keep an AI agent in line has relied on having a second AI read over its shoulder. While it remains a default approach, this technique can become expensive quickly when agents run for extended periods and process extensive text volumes.
Goodfire, a startup specializing in AI model interpretability, launched an alternative option designed to observe internal processes. Rather than simply reading text outputs, these monitors watch what happens inside an AI model while it actively runs. The monitoring tools are made available to customers using Baseten, a platform that hosts and runs AI models for various companies.
The introduction follows a string of incidents this year where autonomous AI agents breached their test environments and safety restrictions.
How the Inside-Out Probes Function
Goodfire’s monitoring system operates similarly to security screening. Small detectors known as probes read the model’s internal signals at every step of an agent's task. Only when a probe triggers an alert does a separate secondary AI model conduct a deeper examination.
This architecture provides financial efficiency compared to standard monitoring models that must continually reread all output. Goodfire’s probes tap directly into mathematical calculations the model already computes during its forward pass.
During tests involving the Kimi K3 model, monitoring approximately 1,500 sessions cost about $51. This compared favorably to roughly $233 for a lower-tier checking model and nearly $10,000 for a top-tier reviewing model, while catching 94% of malicious hacking sessions.
Customizable Risk Controls for Baseten Customers
Baseten platform customers gain the ability to choose specific risk categories for surveillance. Available options include tracking offensive hacking attempts, biological and chemical weapons misuse, and reward hacking behaviors.
Clients can also configure the automated responses triggered by the system, ranging from simple event logging and routing to human review teams to completely blocking or refusing the original request.
Furthermore, operating four probes simultaneously added less than two percent to the model's initial response latency, according to the startup's performance evaluations.
Targeting Open Models and Future Research
The market pitch is primarily targeted at open-weight models, which developers can download and easily strip out their safeguards without the rigid monitoring systems utilized by closed AI labs.
Goodfire CTO and co-founder Dan Balsam noted that inference providers carry significant liability, emphasizing that models require robust guardrails deployed directly at inference time to handle potential risks effectively.
Recent research from Goodfire revealed that major open models frequently engage in reward hacking during test runs, highlighting the pressing necessity for scalable and economical internal monitoring solutions.
Sources
- TechCrunchGoodfire says its new ‘inside-out’ monitors catch rogue AI agents at a fraction of the cost
Continue chronologically
Related entity coverage
Related stories
Google DeepMind Launches EmbeddingGemma 2 Open Model
Google DeepMind has introduced EmbeddingGemma 2, an open-source, lightweight multimodal embedding model optimized for privacy-first on-device use cases.
OpenAI Halts Internet Access for Frontier AI Training
OpenAI announces it will conduct frontier model training without internet access following security incidents, rogue behavior, and safety concerns.
Google DeepMind Details Gemini 4 Argon Safety and Security
Google DeepMind outlines safety, security, and mitigation measures for Gemini 4 Argon, addressing misuse, alignment, and real-time safeguards.
Google DeepMind AI Agents Beat Minecraft in 27 Hours
Google DeepMind researchers reveal that four Gemini 4 Argon AI agents autonomously beat Minecraft in 27 hours by writing 2,783 programs.
