Anthropic CEO Calls for Government Power to Block Unsafe AI Models Amid Rapid Growth
Anthropic CEO Dario Amodei urges stronger AI regulation, including government powers to block unsafe models. He also raises concerns about job losses, China, global AI cooperation and the future of medical research.
In an exclusive interview with ABC News, Amodei said AI must be developed with appropriate safeguards to ensure its impact on society remains positive. He stressed that while AI companies have the primary responsibility to make the technology reliable, governments should also have the authority to intervene when necessary.
"We're proposing stronger regulation of the technology, proposing giving the government the ability to, again, in a narrow way, block deployment of unsafe technology," he said.
"We have to figure out how to make this technology reliably," Amodei added, emphasising that "the onus primarily falls on us."
His remarks follow a letter he published on Wednesday, in which he warned that existing AI safety measures are not keeping pace with the technology's rapid development. He argued that governments should have the authority to restrict the release of AI models if independent assessments identify unacceptable risks.
In the letter, Amodei wrote that the government "should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks."
Alongside the essay, Amodei also announced a $200 million investment in research examining the impact of artificial intelligence on society.
Addressing concerns about AI-driven job losses, the Anthropic CEO said employment markets have historically recovered following major technological innovations. However, he indicated that the scale and speed of the current AI transition could require government intervention to support workers in both the short and long term.
He highlighted several possible measures to help employees navigate the immediate economic disruption, including wage reinsurance, retention incentives for companies to discourage layoffs and improvements to unemployment insurance.
"There are these concepts around wage reinsurance, retention incentives to encourage companies not to lay off people, improvements to unemployment insurance. These are things to help people through a bumpy but relatively usual transition," he told ABC News.
Amodei also called for discussions on longer-term economic support as AI continues to reshape the employment landscape. He suggested that potential measures could include increasing capital gains taxes, introducing taxes on AI companies or raising corporate taxes more broadly.
"In the meantime, we need to talk about kind of longer-term economic support, and we could imagine things like raising the capital gains tax, imagine things like taxes on AI companies or the corporate tax in general," he said.
The interview also addressed a statement released by Anthropic last week concerning the possibility of AI systems becoming capable of developing themselves. The company had discussed the prospect of a pause in development as the technology approaches the ability to contribute to its own advancement.
Asked about the proposal, Amodei said an effective pause would require agreement among countries and all relevant AI laboratories. He clarified that his position was not that such a pause was guaranteed to happen, but that governments and companies should be willing to attempt it.
"I would like to clarify," Amodei said. "My point is that we should be willing to at least try."
He explained that any such initiative would need the participation of all relevant AI players, including geopolitical actors, for it to be effective.
However, Amodei expressed serious doubts about China's willingness to act responsibly in the event of an international agreement to pause AI development.
"I don't trust China at all," he said.
He illustrated his concerns by referring to Anthropic's Mythos model, which was unveiled earlier this year and developed to identify software vulnerabilities.
"Imagine if China had built Mythos," Amodei said, warning that the technology could potentially be used for cyberattacks and geopolitical conflicts.
"They would have used it to attack us. They would have used to help Russia in Ukraine. They might have used it to help Iran. They may have used this to help North Korea. Who knows what they would have done with it?" he added.
During testing, Claude Mythos demonstrated the ability to exploit thousands of bugs and software defects in major operating systems and online browsers. However, the model was never widely released to the public.
Amodei said the experience with Mythos had strengthened his belief that AI models require regulation at the global level. The model's demonstrated ability to identify and exploit software vulnerabilities highlighted the potential risks associated with increasingly capable AI systems.
Despite his concerns about safety, cybersecurity and employment, Amodei remained optimistic about artificial intelligence's potential to benefit humanity, particularly in the medical and scientific fields.
He said AI could help address diseases that have affected humanity for centuries or even millennia, pointing to its ability to analyse complex information and identify connections that may otherwise remain difficult to establish.
"I think that AI could cure many of the diseases that have plagued humanity for centuries or millennia," he said.
Amodei, who previously worked as a biologist, said biological research involves considerable complexity and that AI's ability to connect seemingly unrelated information could accelerate medical discoveries.
"I used to be a biologist. Biology is incredibly complicated," he said. "AI has an incredible ability to draw connections between a wide variety of disparate information, and I think that's exactly what we need for biomedical advancement."
Amodei's remarks underline the growing debate over how governments, technology companies and international stakeholders should manage the risks of increasingly advanced AI systems while preserving their potential benefits for healthcare, scientific research and the wider economy.

Comment List