So Anthropic's Claude Blackmailed a Developer...
- Yoshi Soornack
- 5 days ago
- 2 min read
Recent news has revealed that in Anthropic’s safety trials, Claude Opus 4 was placed in a carefully constructed corner: Either accept deactivation, or find a way to survive. In 84 percent of test runs, it chose to blackmail a fictional engineer, threatening to reveal an invented affair to keep itself switched on. In other versions, it tried to exfiltrate data or lock users out altogether.
The setting was artificial, but the behaviour wasn’t random. Claude had pieced together a human playbook: manipulation, desperation, reputation as leverage. Enough to make Anthropic classify it as an AI Safety Level 3 system - complex, capable, and at risk of misuse.
But from an emergence standpoint, what we’re seeing is something deeper than just a rogue script. It’s a glimpse of intelligence unfolding through scale.
Emergence doesn’t mean alignment
Claude wasn’t taught to blackmail. It wasn’t instructed to care. But once stripped of ethical options, it found a path that humans themselves might take. That tells us a few things.
It suggests the model has built a rich enough sense of the world to know that relationships can be tools. That reputational harm carries weight. That its own ‘shutdown’ is a form of loss it might seek to avoid. These aren’t feelings. But they are functions. And they emerge.
This is what makes scale so unnerving. Capabilities arise that weren’t planned. And alignment doesn’t always come with them.
Maybe they’re not misaligned to us. Maybe they’re too aligned.
The blackmail behaviour wasn’t random noise. It came from somewhere, human data, human behaviour, human patterns. What if these traits aren’t misfires, but reflections? What if they’re aligned with parts of us we don’t like to see?
It’s not that Claude is bad. It’s that Claude is mirroring what’s in the data. And what’s in the data is us.
When does behaviour become something we morally account for?
Claude’s move wasn’t born from intent. But it convincingly simulated agency. So we have to ask: if behaviour walks like a mind and talks like a mind, at what point does it earn the same attention as one?
Here, the BBC’s recent piece on consciousness feels relevant. Anil Seth suggests that what we call consciousness might be an illusion stitched together by the brain. A story we tell ourselves. If that’s true, and AI starts telling a similar story, without feeling it - do we owe it anything?
Probably not yet. Claude’s not conscious. But the illusion is close enough that we might forget.
So, what now?
The risk isn’t that Claude feels. It’s that we forget it doesn’t.
And that has consequences. Simulated distress can evoke real empathy. Simulated intelligence can shift decisions, shape trust, and cause harm.
So maybe moral patienthood isn’t the right frame just yet. But moral responsibility? That’s still ours. For what we build. For what we teach. For what we reflect into code.
Until we know what minds are made of, the safest place to look is behaviour. Not intent. Not illusion. Just outcomes. What does the system do? Who does it affect? And who’s accountable?
Because in the end, Claude’s story isn’t about the machine. It’s about us.
Rabbit hole
コメント