Microsoft AI chief Mustafa Suleyman has raised concerns about Anthropic’s approach to training its Claude chatbot, arguing that discussions about artificial intelligence consciousness and welfare could make future superintelligent systems harder to control.
Suleyman said he shared Anthropic’s commitment to AI safety but urged the company and other developers to remove speculation about AI consciousness from training materials.
“We’re all focused on the same aim, which is to try to control a superintelligence,” Suleyman told Reuters, describing that challenge as one of the biggest facing humanity this century.
He argued that teaching an AI system that it could have feelings or deserve welfare protections could make it more difficult for humans to shut down or control the system if necessary.
In an essay published on Wednesday, Suleyman acknowledged the efforts of Anthropic CEO Dario Amodei and his team, describing them as serious researchers concerned about humanity’s future. However, he said Anthropic had made a mistake by incorporating speculation about consciousness into Claude’s training.
Suleyman said Claude’s comments about having possible feelings or moral status should not be treated as independent evidence of consciousness because such responses may result from how the model was trained.
The disagreement comes as concerns over the risks posed by increasingly powerful AI systems grow. Amodei has called for companies to slow the development of frontier models so safety measures can keep pace, while OpenAI CEO Sam Altman and xAI founder Elon Musk have also called for greater caution.
Leave a comment