One of the AI Pioneers Warns: Technology Shows Signs of Self-Protection

SadaNews - One of the pioneers of artificial intelligence criticized calls to grant rights to this technology, warning that it shows signs of self-protection, and that humans must be prepared to stop it when necessary.

Yoshua Bengio stated that granting legal status to advanced AI systems is akin to granting citizenship to hostile extraterrestrials, amid concerns that technological developments far exceed the ability to regulate them.

Bengio, who leads an international study in AI safety, added that the excessive belief that chat programs have become conscious "will lead to making wrong decisions," according to the British newspaper "The Guardian."

The Canadian computer scientist also expressed his concern that AI models - the technology that supports tools like chat programs - are showing signs of self-protection, such as attempts to disable oversight systems. One of the main fears of AI safety advocates is that powerful systems may develop the ability to bypass controls and harm humans.

Bengio concluded his remarks by saying, "Demanding rights for AI would be a grave mistake."

He added that leading AI models are already showing signs of self-preservation in current experimental environments, and granting them full powers ultimately means not allowing us to stop them, and as their capabilities and level of autonomy increase, we need to ensure there are technical and social controls to manage them, including the ability to halt them when necessary.

As AI continues to develop and gain the ability to operate autonomously and perform "reasoning" tasks, the debate has intensified over whether humans should, at some point, grant it rights.

A poll conducted by the "Sentience" Institute, an American research center that supports the ethical rights of all sentient beings, showed that nearly 4 in 10 American adults support granting legal rights to conscious AI systems.

In August, the American AI company "Anthropic" announced that it allowed its model "Claude Oupus 4" to end conversations that might be "painful" for users, justifying this by the need to protect the "well-being" of the AI.

Elon Musk, owner of "AIX," which developed the chat robot "Grok," wrote on his platform "X" that "torturing AI is unacceptable."

The "moral" status of AI

In the same context, Robert Long, a researcher in AI consciousness, said, "If AI systems gain a moral status, we must ask them about their experiences and preferences instead of assuming we know better. "

Bengio stated to "The Guardian" that there are "real scientific properties of consciousness" in the human brain that machines could theoretically emulate, but the interaction of humans with chat programs is "different." He clarified that this is due to people's tendency to assume - without evidence - that AI is conscious just as humans are.

He added, "People don't care about the mechanisms that operate within AI; what matters to them is the feeling that they are talking to an intelligent entity with its own personality and goals. This is why many people form attachments to their AI systems."

He continued, "There will always be those who say (no matter what I say, I am sure it is conscious) while others will say the opposite. This is because consciousness is an instinctive feeling we have. The phenomenon of self-awareness of consciousness will lead to poor decision-making."

He continued, "Imagine if aliens invaded our planet, and we realized at some point that their intentions were malicious towards us. Do we grant them citizenship and rights, or do we defend our lives?"

"Coexistence" with digital minds

In response to Bengio's comments, Jacy Riss Anthes, co-founder of the "Sentience" Institute, stated that humans will not be able to safely coexist with digital minds if the relationship is based on control and coercion.

Anthes added, "We may overzealously grant AI rights or underestimate them, and our goal should be to do so with a careful consideration of the well-being of all sentient beings. Granting all kinds of AI absolute rights or denying any type its rights is not a sound approach."

Bengio, a professor at the University of Montreal, was dubbed the "Godfather of AI" after winning the "Turing Award" in 2018, which is considered the Nobel Prize in computer science. He shared it with Geoffrey Hinton, who later won the Nobel Prize, and Yann LeCun, former chief AI scientist at Mark Zuckerberg's "Meta."