Anthropic, the company behind the Claude models, is lobbying for its own draft federal AI safety law in the US, Politico reports. The core idea is that if a model poses a direct and foreseeable risk of catastrophe, a court could order its distribution to be paused.
The bill sets a concrete threshold for a catastrophe: a risk of more than 50 deaths or over $1 billion in damage. It covers three kinds of threats: help with building biological weapons, a model escaping its developer's controls, and cyberattacks.
Who decides
Model risk would be assessed by independent experts registered with the US Department of Commerce. AI companies could pay them, but the fee must not depend on the outcome of the review. If the experts find an imminent catastrophic threat, they would have 72 hours to notify the department.
Why a court
Under current rules, no federal agency can ban the distribution of an AI model on its own. So the scheme works in two steps: Commerce receives the experts' warning and asks a judge in Washington to pause the model's release. The thresholds of 50 lives and $1 billion mirror California's SB 53, which also labels such scenarios a catastrophic risk.
What about state laws
There is a second part to the bill: it would override some state rules on risk disclosure, third-party model audits and incident reporting. Tech companies have long complained that complying with dozens of different local laws is hard. Critics may see this as an attempt to replace stricter local requirements with a single rule that suits the industry better.
Our take
It is telling that the limits are being proposed by a model developer itself, which lets it shape the framework it is willing to live with. For anyone building bots and services on third-party APIs, it is a reminder that access to a model may depend on regulators as well as providers, so a fallback to another model is worth having. Details are in the Habr report.