Artificial intelligence company Anthropic has revised its usage policy to prohibit users from engaging in sustained and unnecessary abusive or cruel behaviour towards its Claude AI models.
The San Francisco-based company introduced the restriction as part of an update to its usage guidelines, stating that Claude would retain the ability to end conversations involving such behaviour.
“Claude’s ability to end these interactions will remain the primary enforcement mechanism,” the updated policy states.
Although the policy does not explicitly refer to “AI welfare” — the idea that artificial intelligence systems may deserve certain protections typically associated with living beings — Anthropic and its executives have previously discussed the possibility.
The company did not immediately respond to a request for comment.
Anthropic introduced a similar measure last year, giving Claude the ability to terminate conversations in rare and extreme cases involving persistently harmful or abusive interactions.
In February, Anthropic chief executive Dario Amodei told The New York Times that he was uncertain whether AI models could be conscious but remained open to the possibility.
“We don’t know if the models are conscious… But we’re open to the idea that it could be,” Amodei said.
The question of whether artificial intelligence can experience consciousness has long featured in science fiction and public debate. However, some experts argue that determining whether AI systems are conscious may not be the most productive way to approach the issue.
Jackson Stakeman, general manager at Atlanta-based AI services provider Sparq, told AFP that consciousness was difficult to establish even in other humans, making similar debates about AI potentially inconclusive.
He suggested that AI systems reflect human inputs and behaviour on a large scale, arguing that this alone provides a reason to consider policies governing interactions with them.
The concept of machine consciousness remains controversial, with several prominent figures rejecting the idea that current AI systems possess feelings or the capacity to suffer.
Microsoft AI chief Mustafa Suleyman argued in a recent essay that AI systems are not conscious and do not experience emotions or suffering.
He also warned against granting rights or moral protections to technological systems that could eventually become significantly more capable than humans.
Pope Leo XIV has similarly questioned whether machines can possess qualities associated with human existence.
During a sermon delivered in Italian at St Peter’s Basilica in Vatican City on Thursday, the pope said machines primarily process information rather than understand experiences in the way humans do.
He argued that human thought involves more than processing data, emphasising the importance of lived experiences and the deeper meaning people derive from them.
Anthropic’s updated policy adds to the broader debate over how AI companies should manage interactions with their systems as the technology becomes increasingly sophisticated, while questions surrounding machine consciousness and ethical treatment remain unresolved.
