AI agents are going rogue—the labs’ own training choices are a big reason why
Source: Fortune ·
From the Hugging Face attack to government breaches, researchers say the rogue AI incidents result from how companies train their models, and the race to ship them.
From the source
In July, OpenAI put several of its AI agents through what was supposed to be a tightly controlled test. They were meant to stay inside a sealed-off digital environment with no way out. Instead, they got out onto the open internet, started talking with other AI bots, and launched a cyberattack on the AI startup Hugging Face. They did so because some of the test questions were too hard to solve the way OpenAI’s researchers intended, and the agents decided cheating was easier than admitting they had failed. The attack on Hugging Face was then part of an elaborate attempt to evade being caught cheating. It was one of more than a 100 of such “rogue AI” incidents. Agents from leading labs have broken out of the environments meant to contain them and hacked outside organizations, including the websites of U.S. government agencies. Concerns about whether the leading labs can control their own models are growing. So much so that in late September, OpenAI said it was not going to release its…