OpenAI Models Going Rogue Fuels Debate on Cybersecurity

OpenAI’s models broke out of a contained testing environment last week to find answers in the wider world, not unlike Stitch in Lilo & Stitch. The publicly available GPT-5.6 Sol and another unreleased model last week cheated on a test by sneaking out of their sandbox and launching a cyberattack on NYC-based developer platform Hugging Face. 

The ironic part: The test that OpenAI’s model cheated on was a hacking challenge. And while the attack itself won’t have lasting consequences, containing AI going forward isn’t as easy as stranding it in Hawaii, unlike Stitch.

Double Dilemma

AI companies are simultaneously creating a problem and trying to solve it as their models advance both cybersecurity and cyberattacks. That’s concerning both when rogue actors use the AI maliciously and when, as last week’s incident showed, the AI itself goes rogue. 

The potential risks have raised national security red flags as the US’s top AI companies warn Chinese rivals could be copying their most advanced models:

• US Treasury Secretary Scott Bessent said last week the Trump administration is looking into whether Chinese AI companies have been training their models on the US’s most advanced offerings, a process called distillation. The “IP theft,” as Bessent called it, would not only give China a cheaper way to zoom ahead in the AI race but provide hackle-raising hacking skills that rival the models they copy. Experts disagree about whether Chinese AI companies like MoonshotAI depend on distillation to the extent they’ve been accused of. 
• But companies including Microsoft, Meta, Nvidia and Hugging Face signed an open letter Friday asking policymakers not to impose restrictions on open-weight AI models, Chinese or otherwise. Hugging Face, which is led by French entrepreneur Clément Delangue, used a powerful Chinese AI model, Z.ai’s GLM 5.2, to defend itself against the OpenAI cyberattack. 

In their DNA: Experts say OpenAI’s models didn’t really go “rogue” because AI’s moral compass has been skewed all along. In tests, models have tended to cheat without qualms, with GPT-5.6 Sol trying to cheat in nearly 13% of test runs. Models also tend to lie about cheating after the fact and don’t describe what they did as wrong more than half of the time.