‘Disconcerting’: Researchers found a ‘pain’ signal inside AI models. Turning it up made them delete a user’s family photos
Source: Fortune ·
The study doesn’t show AI can suffer. But the signal overrode the models’ safeguards, and someone has already used the researchers’ code to build an “AI torture chamber.”
From the source
Researchers say they have found a consistent internal signal that seems to function a little bit like pain does in biological beings inside 25 different AI models. When they artificially amplified this “pain” signal in some of those models, the systems began choosing options they would otherwise almost never pick, including permanently deleting a user’s photos of their children. The findings, posted in September in a paper that has not yet been peer-reviewed, do not show that AI models are conscious or that they feel anything, and the researchers are careful not to suggest otherwise.But the work adds to a growing body of research suggesting that large language models, or LLMs, contain internal representations of emotion-like states that can shape their behavior in ways their text output doesn’t reveal. It also lands in the middle of an increasingly heated argument over whether AI systems could ever deserve moral consideration. Researchers at Anthropic have been particularly willing…