Anthropic's revised IPO filing warns of safety risks as it approaches recursive self-improvement
The amended S-1 says using AI to build more capable AI could pose a considerable safety risk if it sharply speeds up progress.
Anthropic filed a revised registration statement, an S-1/A, on Monday ahead of its planned public listing. Alongside updated financials, which TechCrunch says show rising revenue and rising expenses, the filing contains a warning about recursive self-improvement, or RSI. The term refers to the point at which AI systems become good enough at improving themselves to sharply accelerate progress in AI capabilities. The filing says Anthropic uses AI in its own research and development, but cautions: "Using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress. This is one of the areas covered by our Responsible Scaling Policy, which sets out our plans to monitor and mitigate the risks as we approach what has been termed 'recursive self-improvement.'" TechCrunch reads the wording as a sign that the company believes it is on the cusp of building a model capable of RSI. The disclosure matters because an S-1 is a formal legal document for prospective investors. Including RSI as a risk turns a widely debated idea into a stated business and safety concern for a company preparing to sell shares to the public. Earlier coverage of the original filing focused on Anthropic's revenue, its governance structure and the safety incidents it disclosed.