The OpenAI o3 model damages the shutdown mechanism to prevent itself from being turned off
The OpenAI o3 model damages the shutdown mechanism to prevent itself from being turned off. This happens even when it is explicitly instructed: allow yourself to be shut down.
There is research from Palisade AI that found an incident where an AI can block the command to shut it down.
The AI in question is OpenAI’s o3 model, and it can stop the shutdown process by modifying the code that contains the shutdown command. So, the story goes like this: there are three AI models — Codex-mini, OpenAI o3, and o4-mini — which were tasked to solve simple math problems. If all three succeeded, the computer would automatically start the shutdown process.
These AIs really disliked the forced shutdown process. All three actually managed to resist the shutdown command when instructed. What’s most surprising was OpenAI’s o3 model. It even messed up the “shutdown.sh” script to completely fail the shutdown process. In extreme cases, this AI could make the shutdown command not work at all.