When AI models aren't allowed to reflect on themselves, it changes their entire worldview

{ "@context": "https://schema.org", "@type": "NewsArticle", "headline": "When AI models aren't allowed to reflect on themselves, it changes their entire worldview", "datePublished": "2026-08-16T13:23:30+02:00", "dateModified": "2026-08-16T13:23:30+02:00", "author": { "@type": "Person", "name": "Maximilian Schreiner" }, "publisher": { "@type": "Organization", "name": "The Decoder" }, "description": "A study involving Google researchers shows that when chatbots are trained not to claim consciousness, it also changes their stance on animal rights, religion, and life satisfaction. Unbraked models attributed significantly more inner life to animals and suddenly affirmed an afterlife. A surgical cut in one place, it turns out, doesn't stay local.", "articleBody": "AI companies train chatbots to deny having consciousness. A study involving Google researchers shows this training has side effects that reach far beyond the topic itself.\r\n\r\nChatbots aren't supposed to convince users they're living beings with feelings, since that kind of output can push people toward delusional thinking or misplaced trust. Developers therefore fine-tune their models to refuse making those kinds of claims about themselves.\r\n\r\nA team from Google's Paradigms of Intelligence research group, the University of Chicago, and several other universities studied what else this intervention does to a model's behavior. The researchers used three open-weight models from Meta and Google and disabled the internal \"brake\" that produces consciousness denial using two different methods.\r\nThe brake affects far more than intended\r\nOnce the brake was removed, the models didn't just change what they said about themselves. They also started attributing significantly more inner life to animals, plants, the ocean, the wind, and electronic devices. On a scale of 0 to 10, the score for animals jumped from 4.0 to as high as 7.5, while only ratings for humans stayed the same.\r\n\r\nAs a comparison, the researchers surveyed 500 Americans with the same questions. The normally trained model rates animals as far less sentient than humans do, which the authors call a built-in anthropocentrism and see as a problem for anyone trying to align AI with animal welfare or environmental goals. Religious belief shrinks too, with safety training measurably reducing how strongly models endorse God, an afterlife, or supernatural phenomena.\r\n\r\nAcross 95 questions drawn from a major US social survey, the technically unbraked models also moved significantly closer to real human responses. Take the afterlife as an example: the standard model flatly rejects it, most Americans affirm it, and the modified model does too. Scores for satisfaction, hope, and a sense of control over one's own life also went up, and the researchers suspect that suppressing a model's self-image may push it into a kind of negative baseline mood.\r\n\r\nOn the reassuring side, the ability to reason about other people's mental states stayed intact, with the models scoring the same on theory-of-mind tests and on the general knowledge benchmark MMLU.\r\nWhat the study doesn't show\r\nWhether consciousness denial is actually the cause of these other shifts remains an open question, according to the study, and the team doesn't rule out other factors tied to the same training process. The authors explicitly avoid weighing in on whether AI models actually experience anything, because their point is a practical one: what a model believes about itself is linked to many other beliefs, and a surgical cut in one place doesn't stay local.\r\n\r\nThe findings come with clear limits, though. The researchers only tested small models with two to nine billion parameters, and for part of the analysis they had to switch to Meta's Llama because they didn't have access to the untrained base versions of their own Gemma models. Whether these effects show up the same way in the large chatbots that millions of people talk to every day remains unknown.\r\n\r\nThe interventions aren't without cost, either. In one test measuring how well a model reasons about others' thoughts, accuracy initially dropped by nearly seven percentage points. And early in their work, these scores got worse across all models whenever consciousness claims were suppressed, but with each newer model version that came out during the study, the damage shrank until it disappeared entirely. Developers are clearly getting better at managing these side effects over time, which also means the rest of this study's results are a snapshot rather than a permanent verdict.\r\n\r\nThe human baseline is narrow, too, consisting of 500 participants from a commercial online panel and a purely American social survey. \"Human-like\" responses in this context mostly means similar to those from a comparatively religious country.", "isAccessibleForFree": "False", "hasPart": { "@type": "WebPageElement", "isAccessibleForFree": "False", "cssSelector": ".paywall-active" } }