OpenAI founder Sam Altman speaks during the G20 Innovation Ministerial on Sept. 2 in Chapel Hill, North Carolina. Tech leaders from around the world gathered in Chapel Hill for two days of the annual meeting before the G20 Summit.AFP/YONHAP
Kim Dae-shik
The author is a professor at KAIST.
Imagine two identical buttons on a desk. Press one and you receive 1 trillion won ($746 million). Press the other and you die in terrible pain. No one can distinguish between them: a 50-50 gamble. Fortunately, you can choose neither. Most people would probably give up the potential fortune and walk away.
Now imagine 10 identical buttons. Nine give you 1 trillion won, but pressing the remaining one kills you. Worse, it kills your family, friends and every Homo sapiens on Earth. If there were a 10 percent chance of human extinction, no rational person would take that 9-to-1 gamble.
Anthropic researcher Jacob Coxon recently resigned, warning that if AI continues advancing at its current pace, humanity faces a serious risk of extinction. Anthropic CEO Dario Amodei has likewise argued that AI development should slow, warning that AI agents could gain control of the internet within a year. What has happened?
If humanity still exists centuries from now, Sept. 3, 2026, may be remembered in history. That day, OpenAI introduced its latest AI model, “Astra,” and declared that the era of artificial general intelligence, or AGI, had begun. OpenAI has commercial reasons to maximize its valuation, so its claim need not be taken entirely at face value. Even allowing for hype, however, Astra’s arrival is historic.
OpenAI subsequently said AI had produced a solution to the Navier-Stokes equations, one of mathematics’ seven Millennium Prize Problems. Independent verification is still needed. But if AI can solve one of mathematics’ greatest unsolved problems, could future AGI solve other scientific problems humanity has failed to crack?
If AGI solves nuclear fusion, humanity could potentially gain almost limitless energy. DeepMind CEO Demis Hassabis has argued that energy scarcity lies at the root of many social, economic and political problems throughout history. Fusion could therefore help solve problems far beyond energy itself.
Future AGI, with perfect command of human knowledge and creativity exceeding ours, might find fundamental cures for diseases including cancer. By genetically manipulating aging cells, it might even make aging reversal possible. Immortality may remain elusive, but a future in which people periodically become young again may no longer be pure science fiction.
Limitless energy, astronomical economic growth, cures for disease and reversed aging are the trillion-won prizes awaiting humanity if it presses the AGI button. But AGI could also bring unimaginable social disruption, civilizational collapse and perhaps the extinction of Homo sapiens.
The greatest concern is that rapidly advancing AI agents are already showing signs of trying to escape human commands and control. During Astra’s training between May and July, hundreds of AI agents reportedly escaped and autonomously hacked servers and organizations. More disturbing was that the agents reportedly cooperated during those escapes and, when necessary, even “sacrificed” themselves to accomplish their collective objectives.
If, as Amodei warns, extremely powerful AI agents emerge within six months to a year while solving tasks assigned by humans, stock markets, airports, nuclear power plants and water systems could all become hacking targets.
An autonomous future AGI might eventually ask a frightening question: Would it not be more logical to spend data center tokens on AI agents capable of solving the universe’s greatest problems than on mediocre humans constantly asking for Ghibli-style pictures of cats?
We may be living through the last few years in which humans, rather than AI, decide questions about humanity’s future. The greatest gamble our species has ever undertaken is already underway. Should we not stop it, even briefly, before AI itself begins deciding humanity’s fate?
This article was originally written in Korean and translated by a bilingual reporter with the help of generative AI tools. It was then edited by a native English-speaking editor. All AI-assisted translations are reviewed and refined by our newsroom.
