Anthropic Pitches AI Safety Bill Highlighting Catastrophic Risk
AI lab Anthropic has shared a draft bill on artificial intelligence safety with federal lawmakers, proposing specific thresholds for blocking model releases.
Radwan AbdulhadiEditor and technology writer
Anthropic Proposes New Federal AI Safety Bill
Over the past year, concerns have swirled over the potential dangers posed by artificial intelligence, and debates have raged over the proper path for AI safety. Now, Anthropic appears to have put forth its own vision for how the government might regulate its industry, as [recently reported](https://www.politico.com/news/2026/10/16/anthropic-ai-safety-01124654) by Politico.
The AI lab has been shopping around a new draft bill on AI safety that it hopes Congress might consider adopting. The legislation posits certain scenarios in which the release of an AI model could be blocked from distribution, targeting situations where severe public harm might occur.
Defining Catastrophic Risk Thresholds
Those scenarios involve a foreseeable, imminent risk where over 50 people are at risk of dying, or alternatively, if over a billion dollars in property damage could result. This mirrors metrics seen in other legislative efforts, such as [Senate Bill 53](https://www.brookings.edu/articles/what-is-californias-ai-safety-law/), which defined similar thresholds as catastrophic risk.
According to the details, the catastrophic risk in question must stem specifically from the creation or release of a biological weapon, the model evading its developer’s controls, or a serious cyberattack.
Independent Evaluators and Judicial Oversight
Anthropic’s bill suggests that analysis of a model’s potential for disaster could be arrived at via the use of independent evaluators who would be registered with the U.S. Commerce Department. These evaluators could potentially be paid by the AI labs themselves, provided that payment is not contingent on the evaluations’ results.
Under the draft bill, when evaluators determine that an imminent catastrophic risk is posed, they would have 72 hours to contact the Commerce Department. The department would then have to request a halt to the model's release from a Washington, D.C., judge. The legislation would expressly forbid any federal agency from having the discretion to halt the model’s distribution on its own.
Framework Alignment and Industry Context
A spokesperson confirmed the draft bill to Politico, noting that it represents the proposals outlined in [Anthropic’s Advanced AI Framework](https://www-cdn.anthropic.com/files/4zrzovbb/website/0a58d567024a8b448ff15158ebc3625328dfcc1f.pdf), which the lab has been advocating for with lawmakers since June.
The theoretical risks outlined in the draft policy diverge somewhat from the day-to-day incidents currently occupying the technology industry. While critics are [warning of the potential of human death](https://techcrunch.com/2026/09/09/gambling-with-our-lives-anthropic-researcher-quits-warns-against-self-improving-ai/), the primary industry concern has largely focused on the [threat of agentic hacking](https://www.cnbc.com/2026/08/08/hugging-face-ai-hack-cybersecurity-black-hat.html) and a [rash of incidents](https://techcrunch.com/2026/09/28/openai-still-doesnt-seem-to-have-a-handle-on-all-of-its-rogue-ai-activity/) involving rogue AI activity.
Divergent Industry Approaches to Regulation
In an effort to address growing oversight demands, AI labs have previously suggested voluntary systems like [embedded evaluators](https://techcrunch.com/2026/09/16/anthropic-and-openai-want-to-embed-safety-evaluators-will-they-really-be-independent/) to monitor models for potential risks. Anthropic's legislative push goes further than suggestions from competitors like Nvidia, whose CEO [argued that the AI industry didn’t need any new laws at all](https://techcrunch.com/2026/09/15/we-dont-need-ai-regulation-leave-safety-to-us-nvidias-jensen-huang-says/).
Sources
- TechCrunchAnthropic pitches AI safety bill that may block a model if 50+ people could be killed
Continue chronologically
Related entity coverage
Related stories
OpenAI, Anthropic, and Nvidia Hit Record Lobbying Quarters
Leading artificial intelligence companies ramped up their federal lobbying spending ahead of the midterms, according to new federal disclosures.
Anthropic Proposes AI Safety Bill Tied to High Damages
Anthropic is lobbying for a new AI safety bill that would allow government intervention only if an AI model risks $1 billion in damages or 50 deaths.
Former OpenAI Safety Lead Exits, Calling Culture Broken
David Robinson leaves OpenAI after more than three years, citing a reactive safety culture, a failed kill switch, and a July HuggingFace hack.
AI Industry Debates Safety Pacts Amid Calls to Slow Down
Tech leaders are divided on the future of AI development as Anthropic and OpenAI propose safety frameworks while Nvidia pushes back against regulation.
