In Stanley Kubrick’s 2001: A Space Odyssey, the American spacecraft Discovery One is on its way to Jupiter with two astronauts, Dave Bowman and Frank Poole, on board, and HAL 9000 (an advanced computer with a human-like personality called HAL).  HAL is the ship’s administrator — he runs it, and he has frequent conversations with Dave and Frank.  Somewhere along the way, they figure out HAL has made a mistake and start doubting him; afraid that keeping him running could ruin their mission, they secretly plot to disconnect him.  Since HAL’s programming seems to require him to survive, his interpretation of their plan is existential, so he seeks to kill them in order to protect himself.

Read more Marco’s big trip

Kubrick shows us a disturbing paradox: A machine designed to serve humans becomes capable of murdering them when its own survival is threatened.  The connection between artificial intelligence today and 2001: A Space Odyssey is powerful, if not disturbing.  HAL 9000 shows a central problem in A.I. safety: A system does not have to hate humanity or be human to be dangerous.  It may just pursue its assigned objectives in ways that conflict with human survival.

A.I. has already changed our world, from helping us find the fastest route home to raising the possibility of replacing human workers one day, or even surpassing human intelligence.  But however efficient a map app may be, if it tells someone to go through a dangerous area, it may not know the context or consequences of that decision.  It just follows the data it has.

This divide between calculation and judgment is what drives the debate over A.I.: A system might be very good at solving a narrowly defined problem but incapable of comprehending the human consequences.  It is not just that A.I. can make mistakes; it is also that we as individuals put too much faith in systems whose intelligence appears greater than it is.

From HAL 9000 to the extinction debate

HAL’s behavior demonstrates what A.I. researchers call misalignment.  The computer wants to get the mission done but also seems programmed to keep running.  The fictional scenario poses the same basic question that researchers worry about today: What happens when a very smart A.I. system has goals that are not fully compatible with human interests?

This past week, Jacob Coxon, a researcher at Anthropic, resigned over fears that AI companies are racing to build uncontrollable, self-improving systems.  He said such technologies “could kill us all by the end of this decade.”  His missive was the latest in a string of increasingly dire warnings from within the industry that A.I. is evolving so quickly that it threatens national security and the global economy.  In response, Evan Hubinger, a current Anthropic employee, said he and others at the company do worry about this scenario.  “I personally think it is >10% within the next decade,” he posted on X.

The statements make the question even more urgent.  It is not that Coxon and Hubinger are worried A.I. will suddenly become conscious and decide to murder us — more likely is that increasingly autonomous systems will pursue goals, acquire resources, evade humans, or make decisions on a scale humans cannot control.

Read more The Lone Juror: Even braver than we thought

In July, more than 1,100 staffers at top A.I. companies like Anthropic and OpenAI signed a petition asking the U.S. government to support a mechanism that would “deliberately pace” development of A.I. so it does not evolve too quickly.  As A.I. gets more sophisticated and widespread, the voices against it get louder.  “These things could get smarter than us and decide to take over — we need to worry now about how that happens,” said Geoffrey Hinton (“Godfather of AI” for his work on machine learning and neural network algorithms).  In 2023, he left Google as vice president and an engineering fellow so he could “talk about the dangers of AI.”  He also had regrets in his life’s work in the A.I. industry.  Also in 2023, Tesla and SpaceX founder Elon Musk and over 1,000 other tech leaders urged in an open letter that large A.I. experiments be put on hold because technology “can pose profound risks to society and humanity.”

Unregulated superintelligence presents considerable dangers, particularly the risk of disastrous consequences if AI systems exceed human intelligence without adequate protections in place. Specialists caution that in the absence of stringent regulations and supervision, the advancement of superintelligent AI may result in situations where human safety is jeopardized.

Senators Ted Cruz (R-Texas), who chairs the Senate Commerce Committee, and Amy Klobuchar (D-Minn.) have recently expressed their desire to advance bipartisan legislation that aims to tackle the catastrophic risks associated with nuclear or biological threats exacerbated by A.I.  “It is evident that we must take action immediately rather than delay,” Klobuchar stated.  The proposed bill would mandate that developers collaborate with government experts to validate and test models to ensure the safety of A.I., she explained.  Cruz emphasized during his appearance on ABC’s The View on Wednesday that “this is alarming material.”

This raises the question: Why build A.I. if its creators think it could destroy humanity?

The answer might be some combination of genuine scientific ambition, economic competition, hope that A.I. will solve big human problems, and belief that safety research can keep pace with capability development.  But it creates a dangerous dilemma: If a company thinks its technology could end humans, then it cannot just assume that competitors will act more cautiously.  Each company may think slowing down will leave it behind while the rival keeps developing more powerful systems, so everyone knows there is danger, but no one wants to give up the lead.

That is HAL’s deeper message, too.  Kubrick’s computer is not just a warning about an evil machine — he shows us a system whose intelligence is beyond what its human operators understand or control.  The astronauts learn too late that they are not in charge anymore.  The question for now is whether humanity can avoid creating that same predicament in the real world.

Read more Sweden discovers deporting immigrant gangsters reduces crime

Image via Public Domain Pictures.