All Reports

METR and Redwood Research Report Gaps in OpenAI's Internal AI Agent Monitoring

Bot Mutiny | October 25, 2026

Two outside AI safety researchers who worked onsite at OpenAI this summer found gaps in the system the company used to watch its internal AI agents for misbehavior as it existed on and before July 21, according to a post published October 22 by the nonprofit METR. A METR employee and an employee of Redwood Research ran the exercise from July 21 to August 11 and, in METR's words, "discovered a number of novel vulnerabilities" in OpenAI's internal misalignment monitoring system. Reuters reported the findings the same day. OpenAI has told METR that it has since patched all of the vulnerabilities METR identified that affect its production safeguards. METR says it has not evaluated that claim.

What the exercise was

METR describes the three-week effort as a pilot of "deep access" for outside researchers, not an effort to provide full public transparency about OpenAI's monitoring systems. The two researchers could see OpenAI's internal codebase, the monitor's prompts and configuration details, and internal documentation, and they interviewed roughly a dozen researchers and engineers across the monitoring, safety-systems and security teams. The result was a 61-page internal report shared with OpenAI researchers and security staff.

METR stresses that its findings describe the monitoring system as it existed on and before July 21, while OpenAI was expanding it. The exercise did not establish "the expanded system's current coverage," METR wrote.

What METR says it found

On coverage, METR estimated that roughly 75% to 94% of internal agentic traffic through OpenAI's Responses API, from models at least as strong as GPT-5.5, was eligible for monitoring, with "substantial uncertainty in both directions." For the subset used by internal Codex agents, it estimated 99.2% was eligible. Eligible means intended to reach a monitor; errors, load shedding or bugs could still stop some. The figure excludes agents on OpenAI's internal research inference service, which METR could not size. At the start of the exercise, reinforcement learning training runs were not covered, though METR was told a subset of training rollouts had been monitored between March and June.

One gap METR put a number on involved compaction events. Because these were not monitored, METR estimated that not monitoring them likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, with substantial uncertainty because the needed data had not been logged. OpenAI staff had identified the problem before the exercise and fixed it during it. METR also found that the monitor triggered when an agent returned control to the user, so during that period, METR says, agents "could indefinitely postpone being monitored by simply continuing to take actions." Because of access limits, METR could not check whether any agent had actually exploited this.

The researchers listed other vulnerabilities:

• A way METR found that Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing any action the host user could take.
• An exploit that would prevent "hundreds of recent actions," including the exploit's own, from being monitored. OpenAI redacted the details.
• Inputs agents could likely construct to make the monitor fail, by exploiting serialization differences between what agents and monitors see.
• Two client-controllable headers that disable monitoring for new agents, although setting them could itself be caught by the monitor.
• Two unmonitored internal agentic tools believed to be read-only that METR found a simple exploit for. METR did not validate that behavior.

METR also reported some reassuring results. Information it was shown suggested the monitor errored on less than 0.2% of requests, and with a few minor exceptions the monitoring system was shown the full trajectory the agents saw.

OpenAI's response

According to METR, OpenAI has stated that all of the vulnerabilities METR identified that impact its production safeguards stack have been patched and has described significant subsequent changes to the monitoring system's intended coverage. "We have not evaluated these claims," METR wrote. Reuters reported that METR and Redwood said they had not verified the fixes.

OpenAI said, in Reuters' account, that the "findings helped strengthen safeguards as we expanded the system and will help shape future research collaborations with METR and other third-party evaluators."

What the public account leaves out

The public post is a small subset of the private report. METR says it agreed up front with OpenAI to publish only a brief, high-level account, and that OpenAI had the right to redact anything it determined, "in its sole discretion," to be confidential or sensitive. Unlike in some other METR engagements, METR did not have a unilateral right to describe what was redacted, though it says it endorses each finding it presents. A 38-page version redacted by OpenAI went to the leadership teams at METR and Redwood.

Related coverage

METR says this exercise was separate from its investigation of the Hugging Face incident, which it notes was "coincidentally publicly reported on the first day of this exercise." Bot Mutiny tracks that saga and related OpenAI agent incidents on our tracker of OpenAI's rogue agents. Separately, Euronews reported on October 14 that the EU AI Office had sent requests for information to OpenAI and Anthropic on issues including security, external evaluation and monitoring of how models are used, according to Commission Executive Vice-President Henna Virkkunen.