Anthropic's $1.3 trillion IPO and the AI safety paradox

Anthropic, the artificial intelligence company that created the Claude chatbot and has built its brand almost entirely on warnings about the existential risks of artificial intelligence, is set to make its stock market debut at a valuation of approximately 1.3 trillion dollars — the largest initial public offering in history. The BBC reported that the company is proceeding with the flotation despite having warned in its own regulatory filings that it may be building “one of the most transformative and potentially dangerous technologies in human history.” Those words appear verbatim in the company’s S-1 filing. The same document seeks to raise capital at a multiple that implies Anthropic is more valuable than most European economies. It is, by any measure, one of the more remarkable documents in the history of corporate finance.

The received wisdom

The technology press and investor community have largely treated the Anthropic IPO as an unremarkable consequence of the AI boom: capital flows to the most sophisticated AI players, Anthropic has distinguished itself with frontier models and a safety-focused institutional culture, and the valuation reflects anticipated dominance in enterprise AI and the expectation of regulatory influence as governments around the world construct AI governance frameworks. The “safety” branding, on this reading, is a competitive moat rather than a genuine constraint — it differentiates Anthropic from OpenAI and Google DeepMind in Washington and Brussels in ways that may prove commercially valuable. The catastrophic risk language in the prospectus is treated as legally required disclosure, the sort of boilerplate that investors and analysts read once and then set aside. The consensus view is: this is a very large number for a very powerful technology company, and the market has decided the number is approximately right.

A different read

There is something philosophically incoherent about Anthropic’s position that deserves more than a shrug, and the incoherence tells us something important about the state of the AI industry at this particular moment.

The company’s founders — Dario Amodei, Daniela Amodei, and the other OpenAI alumni who left to start Anthropic — have built their public identity around the claim that AI development poses genuine catastrophic risk and that existing development timelines are moving too fast for adequate safety work to keep pace. This is not a position they have articulated casually; it is central to their research agenda, their congressional testimony, and their public communications. Dario Amodei has spoken in terms of AI systems that could, within years, be capable of causing mass casualties or fundamentally destabilising democratic institutions.

And yet the IPO proceeds at scale. The 1.3 trillion dollar valuation requires a growth trajectory that implies Anthropic will build and deploy vastly more capable AI systems faster than any safety research programme could plausibly validate. The incentive structure of being a publicly traded company — quarterly earnings calls, analyst coverage, shareholder return expectations — is precisely the incentive structure that the safety community has argued makes the open development of frontier AI so dangerous. Public companies do not slow down because the technology might be existentially risky; they accelerate because their shareholders require it.

This is the AI safety paradox in its purest form: the people who claim to be most worried about AI risk are also the people racing hardest to build it, funded by investors who are betting that the worriers will get there first. The rationalisation — that it is better to have safety-focused labs at the frontier than reckless ones — has a certain internal logic, but it is also the rationalisation that any talented team of ambitious people would arrive at if they wanted to do the exciting, remunerative thing while maintaining the self-image of the responsible adult in the room.

There is also a regulatory dimension worth examining. Anthropic has been the most active of the major AI labs in cultivating relationships with legislators and regulators in Washington and Brussels. The company’s safety framing has given it access to policymakers that pure commercial actors like Google or Meta find harder to arrange. A public company with a 1.3 trillion dollar market cap and a standing advisory relationship with the major AI regulatory bodies is not a neutral participant in the governance process: it is a participant with an enormous interest in ensuring that any regulatory framework is designed in ways that advantage its architecture and disadvantage its competitors. The history of corporate capture of the regulatory apparatus that ostensibly governs it is long enough that we should not be surprised when it happens again in the AI sector. But we should name it.

Separately, Al Jazeera reported that US non-profit organisations have filed suit against the major AI labs to halt development — a legal effort that will almost certainly fail but that articulates the same structural concern: the entities developing the most potentially dangerous technology in history are also the entities funding the research that assesses whether that technology is safe.

What to watch

Watch the composition of Anthropic’s post-IPO shareholder base — specifically whether activist investors or short-sellers begin to challenge the “safety-first” branding as incompatible with public company growth obligations. Watch whether Claude’s commercial success in enterprise markets leads to a gradual de-emphasis of safety research relative to product development, as happened at OpenAI after its Microsoft partnership scaled. Watch congressional reaction: the IPO will attract scrutiny from both sides, with the right suspicious of corporate safety-washing and the left concerned about AI concentration. And watch whether Anthropic’s safety research actually produces anything that slows its own deployment timeline — because if it does not, the entire safety rationale for its existence will have been revealed as a market positioning exercise.

— J