“Right now, it’s like the AI industry has a gas pedal but it doesn’t have a brake pedal,” Anthropic co-founder Jack Clark told the BBC.
Anthropic recently overtook ChatGPT as the world’s most valuable AI start-up. But Clark has called for a global freeze in AI development, warning that humans risk losing control of the technology. He revealed that 80% of the code that Claude, the company’s chatbot, is operating on was written by Claude itself. And reaching 100% is only a couple of years away.
Clark and his research colleague, Marina Favaro, have suggested that work at Anthropic could undergo “a meaningful slowdown or pause” if other AI tech firms were prepared to do the same. “If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing,” they wrote in a blog post.
What did the commentators say?
Better regulation “would keep AI systems in their lane”, said David Krueger, a specialist in responsible AI, in The Guardian. We should insist companies have “clear and well-scoped purposes” for their AI tools, and “demand evidence that they are fit for purpose”. And they should report statistics and data so that we can see if their product is being used in ways that “deviate from its intended purpose”.
But the “safest, sanest” option is to “stop racing” to make AI smarter. The creation of Moltbook (a forum for AI agents that humans can only observe) is one of the “increasingly alarming warning signs” that “rogue AI agents” could be on their way. “We need to make sure” that rogue AI isn’t “capable of threatening humanity, by agreeing to enforceable, international limits on AI capabilities and AI development”.
There are some hopeful signs in the US. On Tuesday, Donald Trump signed a “much-awaited” executive order to establish a measure of vetting for AI companies, said Politico. It was “messy, muted and far less ambitious than Silicon Valley’s critics had hoped for” but it does mark a “sea change in Washington’s willingness to tighten” AI oversight. The new voluntary process of sharing new models with the US government, so that security risks can be identified and addressed before the technology is released, could “soon pave the way for mandatory vetting, federal pre-approval of advanced AI systems and other regulations”.
Some may think it “meaningful” that Trump is “doing something – anything – about AI”, said The Atlantic, but this executive order is “relatively toothless”. He wants to look like he’s being robust, to “score points” with the public, but, in fact “he is not saying or doing anything substantive at all”. The window for serious government regulation, anywhere in the world, is “rapidly closing”; “hopefully, it is not already gone”.
We’re missing the point, said John Burn-Murdoch in the Financial Times. “AI’s capacity to deliver genuine value has been vastly exaggerated.” In one US study, researchers tracking software developers before and after they adopted AI tools found an initial “explosive” increase in productivity (300% more files created or edited) but, after verification and review, just a 30% “uplift” in the number of releases. These are “powerful new tools” but it’s going to take some time before they can interact with current workflow “processes and structures” without friction or bottlenecks.
What next?
Trump’s executive order is a “good first move in creating a safer tech ecosystem”, said Jen Easterly, former director of the US Cybersecurity and Infrastructure Security Agency, in The New York Times. But a voluntary framework, predicated on mutual cooperation between private companies and the US government, “cannot guarantee” effectiveness. And, let’s not forget, a “principle enshrined in an executive order is only as durable as the administration that issued it”.
For this step to be a positive one, in an American context at least, the legislative branch needs to follow suit. The responsibility of building an AI environment that is “innovative, trusted and resilient” ultimately lies with the US Congress.
Some tech execs want a ‘pause’; the US president wants voluntary vetting – but can anything help keep AI under control?
