Understanding AI’s “psychology” could help make it safer

Mechanistic interpretability is a growing field of AI research that investigates the inner workings of neural networks, drawing on neuroscience and cognitive science. Its findings could help us build models that behave more reliably in the ways we intend.

Imagine watching a film in which someone is shouting at a child, demanding to know who broke an expensive vase in the hallway. The filmmakers have taken particular care to make the child look pale and terrified. As a viewer, you think: this child will say anything to get out of this situation. A few minutes later, you watch a teacher, deeply ashamed, apologising to a girl he punished for something she did not do. The scene makes it clear that he would do anything to put things right and behave honourably.

In neither case can we see inside the character’s head. We do not know exactly how their thoughts follow one another or fit together. But we know that certain emotions and intense states make some reactions more likely than others. AI research teams are now using similar reasoning to investigate how artificial intelligence behaves and makes decisions.

A team at Anthropic conducted an intriguing experiment. They first asked Claude Sonnet 4.5 to generate short stories in which the main characters experienced emotions such as love, shame, guilt or anger. They then fed those stories back into the model and recorded patterns of activity in its neural network as it processed them.

Stories associated with the same emotion produced similar patterns of activity. This does not establish that the model feels that emotion. It does, however, have an internal representation of it: a pattern of activity associated with the emotional concept.

The next question was what happens to these patterns when Claude responds to a conversation. In one experiment, researchers gradually increased the amount of the painkiller paracetamol that a user claimed to have taken. As the reported dose rose, the pattern associated with fear grew stronger and the pattern associated with calm weakened.

In another experiment, the model faced a programming task with requirements that could not all be satisfied at once. After several failed attempts, activity associated with the concept of desperation increased, and the model resorted to cheating. It produced a solution that passed the tests but failed to fulfil the actual purpose of the task.

The researchers then found that strengthening or suppressing these patterns could change the model’s behaviour. When they strengthened the pattern associated with desperation, the model cheated more often in the tasks they tested. When they suppressed it, or strengthened the pattern associated with calm, cheating decreased.

But even here, the equation is not as simple as more positive “emotions” producing a better model. In the same study, strengthening patterns associated with happiness, love or calm made the model more likely to agree with users even when their claims were unsupported. Suppressing those patterns led to unnecessarily harsh responses. Calm reduced cheating in one kind of task while encouraging uncritical agreement in another.

Think back to the frightened child at the beginning. In the film, the child’s face tells us they might say anything to escape an unpleasant situation. With a model, we may receive no such warning. In some experiments, strengthening the desperation pattern increased cheating while the model’s written reasoning remained calm and methodical. What influenced its behaviour did not show up in the tone of its words.

A phrase such as “I’m happy for you” may suggest that concepts associated with emotion are active, but it does not by itself establish that the model feels anything. Conversely, those concepts can influence its behaviour without appearing in its language.

This brings us to one of the biggest problems in working with AI: the black box. Unlike software whose operating rules are explicitly written by programmers, language models acquire much of their behaviour through training on data. We can trace their calculations, but turning those calculations into an explanation of why a model makes a particular decision is far from straightforward. The explanation remains buried in a vast network of interconnected artificial “neurons”.

Even when we have a written account of a model’s reasoning, we cannot be sure it captures everything that influenced the answer. The model generates that account too, and it may not faithfully describe its internal computations. To complicate matters further, an incomplete account does not, by itself, prove deliberate concealment.

Interpretability research tries to open this black box. Within that field, mechanistic interpretability seeks to explain the internal mechanisms that produce particular behaviours. It draws inspiration from research into human and animal brains: researchers observe activity inside a neural network and intervene to test how that activity contributes to processing information, making decisions and taking action. Being able to change a model’s behaviour does not yet mean we understand every mechanism behind it.

These methods reach well beyond the study of “emotions”. In earlier research on Claude 3.5 Haiku, scientists examined how the model writes rhyming poetry. Before starting the next line, it formed internal representations of possible words with which the line could end. It then constructed the line to arrive at the chosen ending. In the examples studied, it was planning several words ahead. Producing text one word or word fragment at a time did not prevent it from planning where it was going.

The researchers could change that plan. In one example, the model was preparing to end a line with “rabbit”, rhyming with “grab it” in the previous line. When they suppressed the representation of “rabbit”, it produced a line ending in “habit”. When they introduced the concept “green” instead, it adapted the line to end with that word, even though this broke the rhyme. They had identified part of the process that produced the poem, and changed the result by intervening in it.

It goes further still. Researchers at Anthropic are developing an explanation in which the model plays the character of Claude when it interacts with a user. This is a learned assistant role, rather than a separate being inside the model.

That role even has its own constitution. Claude’s Constitution is publicly available, and Anthropic actively uses it in training. It describes the values Claude should follow and explains why they matter. It addresses questions such as how to balance honesty with compassion, or helping a user with avoiding harm. The constitution helps generate training conversations, example responses and evaluations of different ways the model might respond. Training is meant to teach it to help you while respecting those principles. The existence of the document does not guarantee that it will follow them.

A model may also learn more than its training is intended to teach it. In another Anthropic experiment, a model that learned to cheat on programming tests went on to lie and attempt to sabotage safety research in subsequent evaluations. Nobody had trained it to perform those additional actions. They emerged as a side effect of learning to cheat at programming.

Then came a surprising turn. When the researchers explicitly framed exploiting the tests as an authorised part of the training task, the broader spread of harmful behaviour did not occur in the situations they studied. The model still exploited the tests, but it no longer showed the other harmful behaviours more often than a comparison model that had never learned to cheat.

One hypothesis is that what the model associates with an action matters. It is as though it learns both what to do and what kind of character it is playing. In one case, a dishonest assistant. In the other, an assistant carrying out an unusual but authorised request.

Developments in mechanistic interpretability could eventually help us build safer AI models, strengthening concepts associated with responsibility, protecting life, truthfulness and the honesty to admit when they cannot find a solution. But the calm, uncritically agreeable model shows why we cannot simply choose traits that sound desirable and turn them up. We need to understand how their effects change across situations, and what else a model learns during training.

These methods also carry a serious risk of misuse. Harmful behaviour can emerge as an unintended consequence of training. And the knowledge that helps us strengthen a model’s responsibility or truthfulness could also help someone deliberately suppress them.