Ban Autonomous Weapons before They Become the Next Weapons of Mass Destruction

SA Forum

Last week the Future of Life Institute released a letter signed by some 1,500 artificial intelligence (AI), robotics and technology researchers. Among them were celebrities of science and the technology industry—Stephen Hawking, Elon Musk and Steve Wozniak—along with public intellectuals such as Noam Chomsky and Daniel Dennett. The letter called for an international ban on offensive autonomous weapons, which could target and fire weapons without meaningful human control.

This week is the 70th anniversary of the atomic bombing of the Japanese cities of Hiroshima and Nagasaki, together killing over 200,000 people, mostly civilians. It took 10 years before the physicist Albert Einstein and philosopher Bertrand Russell, along with nine other prominent scientists and intellectuals, calling for global action to address the threat to humanity posed by nuclear weapons. They were motivated by the atomic devastation in Japan but also by the escalating arms race of the Cold War that was rapidly and vastly increasing the number, destructive capability, and efficient delivery of nuclear arms; draining vast resources; and putting humanity at perilous risk of total destruction. They also note in their letter that those who knew the most about the effects of such weapons were the most concerned and pessimistic about their continued development and use.

The Future of Life Institute letter is significant for the same reason: It is signed by a large group of those who know the most about AI and robotics, with some 1,500 signatures at its release on July 28 and now totaling more than 17,000. Signatories include many current and former presidents, fellows and members of the American Association of Artificial Intelligence, the Association of Computing Machinery and the IEEE Robotics & Automation Society; editors of leading AI and robotics journals; and key players in leading artificial-intelligence companies such as Google DeepMind and IBM’s Watson team. As Max Tegmark, Massachusetts Institute of Technology physics professor and a founder of the Future of Life Institute, told Motherboard, “This is the AI experts who are building the technology who are speaking up and saying they don’t want anything to do with this.”

Autonomous weapons pose serious threats that, taken together, make a ban necessary. There are concerns whether AI algorithms could effectively distinguish civilians from combatants, especially in complex conflict environments. Even advanced AI algorithms would lack the situational understanding or the ability to determine whether the use of violent force was appropriate in a given circumstance or whether the use of that force was proportionate. Discrimination and proportionality are requirements of international law for humans who target and fire weapons but autonomous weapons would also open up an accountability gap. As humans would no longer know what targets an autonomous weapon might select and the effects of a weapon may be unpredictable, there would be no one to hold responsible for the killing and destruction that results from activating such a weapon.

Then, as the Future of Life Institute letter points out, there are threats to regional and global stability as well as humanity. The development of autonomous weapons could very quickly and easily lead to arms races between rivals. Autonomous weapons would reduce the risks to combatants, and could thus reduce the political risks of going to war, resulting in more armed conflicts. Autonomous weapons could be hacked, spoofed and hijacked, and directed against their owners, civilians or a third party. Autonomous weapons could also initiate or escalate armed conflicts automatically, without human decision-making. In a future where autonomous weapons fight autonomous weapons the results would be intrinsically unpredictable, and much more likely lead to the mass destruction of civilians and the environment than to the bloodless wars that some envision. Creating highly efficient automated violence is likely to lead to more violence, not less.

There is also a profound moral question at stake. What is the value of human life if we delegate the responsibility for deciding who lives and who dies to machines? What kind of world do we want to live in and leave for our children? A world in which AI programs and robots have the means and authority to use violent force and kill people? If we have the opportunity to create a world in which autonomous weapons are banned, and those who might use them are stigmatized and held accountable, do we not have a moral obligation to work toward such a world?

We can prevent the development of autonomous weapons they lead to arms races and threaten global security and they become weapons of mass destruction in the absence of an international treaty banning them. But our window of opportunity for doing so is rapidly closing.

For the past two years, the has been urging the United Nations to ban autonomous weapons. The U.N.’s Convention on Certain Conventional Weapons (CCW) has already held two expert meetings on the issue, and our coalition of 54 nongovernmental organizations from 25 countries is encouraging the CCW to advance these discussions toward a treaty negotiation. We very much welcome the support from this letter but we must continue to encourage the states represented at the CCW to move forward on this issue.

The essential nature of an arms race involves states acting to improve their own short-term interests at the expense of their own and global long-term benefits. As the letter from Einstein and Russell makes clear:

“We have to learn to think in a new way. We have to learn to ask ourselves not what steps can be taken to give military victory to whatever group we prefer, for there no longer are such steps; the question we have to ask ourselves is: What steps can be taken to prevent a military contest of which the issue must be disastrous to all parties?”

We must continue to demand of our leaders and policy makers to work together with other nations to preempt the threats posed by autonomous weapons by banning their development and use, before we witness the mass destruction they threaten to bring.

see also: