Jacob Coxon, a researcher who has worked on AI model training at OpenAI and Anthropic, has resigned from Anthropic, citing concerns over the pace of artificial intelligence development and the potential risks posed by increasingly capable systems.
Coxon’s departure was first reported by the Wall Street Journal, which described it as one of the early instances of an Anthropic employee leaving the company over concerns about AI safety.
In a post on social media platform X, Coxon said he had spent the past three years conducting pre-training research at both OpenAI and Anthropic. He criticised the companies for pursuing self-improving AI systems at a pace he described as irresponsible.
Coxon warned that increasingly advanced AI systems could eventually outperform humans across a range of tasks, including cybersecurity and scientific research, while potentially gaining access to significant resources and influence.
He also said some AI researchers and executives privately shared concerns about the possibility of advanced AI posing severe risks to humanity, despite presenting more measured views publicly.
Coxon questioned why researchers continued developing the technology despite such concerns. He said that while some employees at OpenAI had not fully recognised what he described as the broader implications of advanced AI, researchers at Anthropic understood the risks but felt compelled to continue because of competition within the industry.
He urged AI researchers to reconsider the pace of development and called for greater safeguards before companies begin training increasingly autonomous and self-improving systems.
Speaking to the Wall Street Journal, Coxon said he did not want to contribute to an industrywide race to develop AI systems capable of improving themselves, expressing concern that such systems could become difficult to control.
He said terms such as “crunchtime” and “endgame” were increasingly being used by colleagues to describe the development of self-improving AI models.
Coxon was also among more than 1,000 AI researchers who recently signed a statement calling for greater international coordination and the development of mechanisms that could slow AI development if necessary to maintain control over systems capable of improving themselves.
