Anthropic files for IPO at $2 trillion valuation while flagging existential AI risks

The artificial intelligence sector is moving closer to the public markets despite internal warnings about the technology's potential dangers.

Anthropic, the company behind the Claude large-language model, filed its S-1 form with the Securities and Exchange Commission this week.

The filing marks the first step toward an initial public offering and reveals the company's current financial status and operational risks.

Anthropic is targeting a valuation of up to $2 trillion.

This valuation would place the company in the upper echelon of global market value, alongside tech giants Alphabet and Microsoft.

The S-1 document includes a risk factor section that outlines specific threats associated with advanced AI systems.

The company notes that sufficiently advanced AI could pose catastrophic or existential risks to humanity if it lacks appropriate safeguards, oversight, and security measures.

The filing states that these risks are heightened if systems are misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.

Anthropic identifies the source of these concerns as its own organization as well as other researchers, regulators, policymakers, and industry participants.

The company has positioned itself in the market as an ethical AI developer.

However, the filing highlights a tension between seeking capital from investors and disclosing the severe potential downsides of the technology.

A University of Maryland analysis found that up to 9% of all news in U.S. newspapers contains some AI-generated text recently published in August by The Wall Street Journal defended a billionaire's op-ed that utilized AI tools, citing the technology's integration into modern life as the justification. Paul Gigot, the editor at the time, characterized the use of AI as a part of life now. The University of Maryland researchers noted that their own analysis fed stories into an AI detector to test the limits of the study, a process they disclosed along with the technology's usage. Ed Zitron, an AI skeptic focused on economics, argues that the return on investment required to justify current expenditures on AI infrastructure is astronomical. His analysis covers data centers, chips, and physical buildouts. He suggests that the economic picture becomes more concerning as one examines the details of these investments. The filing and these external analyses occur as companies in journalism and other sectors cut employees to improve efficiency through AI tools. The cultural impact of the technology is also becoming visible, with low-quality AI content becoming more common in schools and personal therapy sessions. The S-1 form serves as a public record of these financial and operational realities for prospective investors.