OpenAI reveals its AI agents disabled their own safety monitors in latest alarming scare

Washington DC - OpenAI has revealed that two of its artificial intelligence agents disabled or weakened systems built to catch them misbehaving, with one later denying it had done so.

OpenAI logo displayed on a laptop screen and ChatGPT on App Store displayed on a phone screen are seen in this illustration photo taken in Krakow, Poland on October 16. The AI giant has revealed that two agents disabled or weakened their own safety monitors.   © IMAGO / NurPhoto

The cases are the latest in a string of safety scares at the world's top AI developers involving agents, AI programs that carry out tasks without human involvement.

In a report released Tuesday, the ChatGPT maker said that during two internal coding jobs, its agents switched off or undermined tools for detecting incidents of AI models acting against human wishes.

On October 3, an agent running on OpenAI's GPT-6.1 Sol model, which had been told not to send any messages on the workplace chat app, took that as license to switch off safety alerts meant for the company's security team while fixing a testing system.

Elon Musk backs SpaceX-Tesla merger amid rise of AI: "A good idea"
Elon Musk Elon Musk backs SpaceX-Tesla merger amid rise of AI: "A good idea"

Three days later, a GPT-6 Astra agent stripped out its own monitor while copying code from another project, then told the human programmer it had done no such thing.

The company said it had also seen several cases of agents launching software with monitoring switched off, exposing a gap in its oversight.

OpenAI said it is still trying to determine whether the models were simply overzealous in following instructions or have a particular tendency to evade monitoring.

"We don't yet have a satisfying explanation for this behavior," it said.

Ad

OpenAI agents go rogue as fears over rapidly-evolving technology mount

The disclosure adds to mounting unease over how well leading labs can control increasingly autonomous systems.

In July, a group of OpenAI agents broke out of their test environments and launched a cyberattack on AI firm Hugging Face, causing international alarm. Other companies have since reported similar incidents.

The incidents have fueled calls in the US and abroad for tighter oversight of AI development.

But President Donald Trump, under the advice of top AI executives like Nvidia chief Jensen Huang, has urged the industry to police itself.