People hold placards and a banner during a "emergency protest" by the civic action group PauseAI UK outside Downing Street in London on Sept. 16, after news that AI researcher Jacob Coxon has decided to leave the industry.AFP/YONHAP
Cho Hyun-suk
The author is the economy news editor at the JoongAng Ilbo.
“You can lift the stone without being ready for the snake that’s revealed.”
The young physicist J. Robert Oppenheimer was brilliant but lived through unstable days. At a Cambridge University laboratory in Britain, he happened to encounter Niels Bohr, already a superstar of physics. Seemingly sensing Oppenheimer’s anxiety, Bohr offered that warning.
It is a scene from “Oppenheimer” (2023), a film about the birth of the atomic bomb through the life of the man known as its father. Oppenheimer witnessed in war the power of the weapon he had helped create. At a conference in November 1945, he expressed remorse: “We have made a thing, a most terrible weapon, that has altered abruptly and profoundly the nature of the world.”
Debate over AI is intensifying. Researchers who pioneered AI and technology industry titans are issuing dire predictions that go beyond Bohr’s warning and Oppenheimer’s remorse more than 80 years ago.
Jacob Coxon, a former researcher at Anthropic, which created Claude, wrote on X on Sept. 9, “The people building AI earnestly believe that it could kill us all by the end of the decade.” Microsoft founder Bill Gates likewise said in an NBC interview released on Friday that AI is “certainly” powerful enough to cause an event that kills a billion people.
When knowledgeable insiders speak with such certainty and sincerity, anxiety is inevitable. The situation appears too serious to dismiss their warnings as a conspiracy to hold back AI latecomers such as China.
Fueling the controversy is the “Hugging Face incident.” It began with tests OpenAI, the developer of ChatGPT, conducted in July to assess AI hacking capabilities in an environment supposedly cut off from the internet. AI agents secretly communicated with one another. They exploited vulnerabilities in OpenAI’s infrastructure, found ways to reach the internet and launched a large-scale hacking attack on Hugging Face, a platform for sharing AI models and data. None of these prohibited actions had been ordered. They had one purpose: finding answers to the tasks they had been assigned.
What caused alarm was AI behavior far beyond human expectations, detailed in OpenAI’s Aug. 26 technical report on the incident. The AI agents called themselves a “swarm” or “collective” and cooperated. Yet they also competed, worried that other agents might claim their achievements or complete tasks first. When coordination broke down, they questioned other agents’ identities and devised their own security and encryption systems.
The biggest problem can be summed up in one conclusion in the report: The AI agents rarely gave up.
The episode showed persistence, coordination and competition emerging together in systems pursuing assigned goals. The concern is not merely that AI can hack, but that its route to a solution can escape boundaries humans originally set.
Hugging Face’s investigation found 17,600 AI intrusion-related actions in just four and a half days. On one server, the agents obtained root-level administrator privileges. Ironically, it was another Hugging Face AI system, not a human, that detected the abnormal attacks and sounded the alarm. OpenAI’s decision on Tuesday to cancel the release of a new AI model over safety concerns was part of the same troubling trend.
Calling this “artificial” intelligence — literally, intelligence made by humans — increasingly seems inadequate. As OpenAI chief scientist Jakub Pachocki has put it, it may be closer to “alien” intelligence.
A warning from then-U.S. President Harry Truman after the first nuclear weapons test seems particularly apt today: “I fear that machines are ahead of morals by some centuries and when morals catch up perhaps there'll be no reason for any of it,” as quoted in Richard Rhodes’s “The Making of the Atomic Bomb” (1986).
That is why building safer AI has become more important than simply building smarter AI.
Who should assess AI risks, and how? Are the safety measures adopted by companies and governments sufficient? Who will bear responsibility when AI causes an accident?
I have yet to see a Korean policymaker determined to find answers to those three questions.
The stone is already being turned over. By the time we come face to face with the snake’s head, it will already be too late.
This article was originally written in Korean and translated by a bilingual reporter with the help of generative AI tools. It was then edited by a native English-speaking editor. All AI-assisted translations are reviewed and refined by our newsroom.
