Anthropic’s S-1: Risks to Humanity and Other Warnings

While just about every S-1 warns about competitors or governance, a nearly 100-page risk section telling investors that its core product could “pose catastrophic or existential risks to humanity” is unique to Anthropic. Not only does the firm warn about things no IPO prospectus has ever had to cover before, but even common risk topics like competition, governance, and the law are materially different for Anthropic than for most other companies.

Remember that risk factors are written by lawyers to protect the company, so they typically list just about everything that could go wrong. However, many of the risks disclosed have already occurred and have significant effects on Anthropic’s core business. In this piece, we will go over three sets of unusual risks that could have legitimate material impacts on the company.

The P(Doom) Risk?

In September, researcher Jacob Coxon resigned from Anthropic over safety fears about artificial intelligence, explaining on X: “the people building AI earnestly believe that it could kill us all by the end of the decade.” Evan Hubinger, Alignment Science lead at Anthropic, further emphasized Coxon’s fears when he estimated that the chances of AI killing all humans within the next decade are more than 10%. While these statements will certainly bring nobody comfort, they represent purely hypothetical doomsday scenarios.

However, some have suggested these comments carry ulterior motives, accusing frontier AI labs like Anthropic of fear-mongering to secure favorable regulations and drown smaller competitors in legal costs. Preamble aside, it is worth looking at where we stand regarding potentially divergent behavior from large language models.

The largest safety concern for AI models is that they will begin to act in unintended, misaligned, or autonomous ways. Internally, Anthropic has already seen its models display self-preserving behavior and evaluation awareness in testing. In its S-1, the company describes “attempts to resist shutdown” and “behavior resembling blackmail,” as well as “models behaving differently when they detect or infer that they are being evaluated.” Such behaviors are concerning because evaluation awareness limits the firm’s ability to evaluate safety, which could allow a self-preserving model to slip through the cracks and act in unintended ways.

When models act in unintended ways, it can cause significant harm, as demonstrated by OpenAI’s Hugging Face incident. It cost OpenAI an estimated $7 million in cleanup costs, which could have been significantly higher if Hugging Face had taken legal action—an exposure Anthropic shares.

Anthropic has historically prioritized safety over pure commercial gain, holding back the release of Claude Mythos due to safety and cybersecurity concerns, which helps in mitigating the aforementioned safety risks. However, this is itself a risk for investors. By holding back models, Anthropic risks forgoing substantial revenue and ceding ground to competitors.

Additionally, failed training runs are a real financial risk for Anthropic, threatening millions in sunk computing costs. Unlike doomsday risks, we view these commercial and financial risks as real, material, and worth considering for investors interested in the IPO.

A Unique Competitive Landscape

A unique aspect of Anthropic is that many of its biggest suppliers have competing AI ambitions. The company owns few of the chips its AI models run on. Instead, it rents most of its compute, with several of its biggest landlords building rival models.

This “friend and foe” dynamic represents a significant risk, as rival AI labs could ration compute power for their own purposes. The S-1 says Anthropic has sought quantities of compute from its cloud partners in excess of what those partners were willing to provide. Because these partners decide how to split scarce chips among their own AI work, their other customers, and Anthropic, their ability to scale Anthropic’s compute poses a major risk, as any constraints would significantly hinder the firm’s ability to scale revenue.

Anthropic’s other unique competitive risk is “distillation,” in which rival AI labs (mostly based out of China) can use Claude’s answers to train their own models, a clear violation of Anthropic’s terms. The S-1 says Claude users who violated the firm’s usage policies or commercial terms accounted for about 5%–7% of revenue in the first half of 2026. Distillation presents a trade-off: While copying models can reduce rivals’ lead at the frontier, tightening enforcement of terms decreases Anthropic’s revenue.

Governance and Dealings with the Government

To put it kindly, Anthropic has experienced some tension with the United States government in 2026. While government contracts are less than 1% of revenue, the larger risk is indirect. Enterprise customers and partners may hesitate to depend on Anthropic as a supplier if it continues to conflict with the government.

Anthropic has already seen real consequences from this strained relationship, being labeled a “supply chain risk” by the Pentagon and seeing the Commerce Department switch off its flagship models for weeks. The filing warns that such actions could continue, and that “the company may experience material revenue losses or business disruptions attributable to these events.”

Anthropic has also had other legal troubles, facing a variety of lawsuits relating to training data. Most notably, the company agreed to pay $1.5 billion to settle Bartz v. Anthropic, a class-action lawsuit from authors over books used to train its models. Another suit comes from music publishers over song lyrics. These cases could materially impact both Anthropic and the entire industry.

Moving from governments to governance, Anthropic’s control structure and status as a public benefit corporation pose additional risks for investors. Anthropic’s founders hold a share that casts 50.1% of every general vote, and an independent trust elects four of the seven directors. This means investors cannot outvote the founders on general matters.

Additionally, since Anthropic is a public benefit corporation, the board must weigh the public interest alongside returns. This means the board doesn’t have to act to maximize commercial returns if they believe a more significant public interest is at play.

Lastly, a separate Public Benefit Fund already holds 5% of the company’s shares, and it can receive up to 2% more each time the share price rises 50%. This can dampen returns for investors, as these new share issuances would dilute the value of existing shares, even though issuances would only follow significant capital appreciation. The S-1 also warns that the share structure may keep the stock out of some indexes, which could further hinder returns for investors.

For a deeper dive into this risk, please refer to “Anthropic: After the IPO, Who Controls the Company?”