Deepfakes, real votes: Officials contend with rise in AI campaign videos ahead of June 3 election

The surge in political deepfakes has become a growing challenge for election authorities ahead of the June 3 local elections.

While the Public Official Election Act bans producing, editing and distributing realistic AI-generated campaign materials within 90 days leading up to elections, officials say the rapid spread of manipulated content — often shared in closed online spaces — is exposing the limits of enforcement and raising broader questions about how to regulate AI in future elections.

Outside the 90-day period, such materials are allowed only if they are clearly labeled “AI-generated.”

Controversy arose on May 13 after a photo of the back of a man in a Rebuilding Korea Party jacket, carrying shopping bags as he walked through a traditional market, was uploaded on social media. 

The figure appeared to resemble Cho Kuk, who is running in a parliamentary by-election in the Pyeongtaek-B constituency in Gyeonggi, but the image was later revealed to be AI-generated. Cho’s campaign denied involvement, saying, “The campaign did not make it.” By then, however, the image had already been viewed thousands of times online.

In another case in February, former Democratic Party preliminary candidate Park Seong-jin, who ran for mayor of Nam District in Ulsan, uploaded a fake news-style video to social media claiming that news magazine TIME had selected him as “the person to lead Nam District, Ulsan.” Both the content of the report and the anchor’s face and voice were fabricated using AI.

The National Election Commission (NEC) filed a complaint against Park on Feb. 9 for distributing false information using deepfake technology, and Park later withdrew his candidacy on Feb. 23. The complaint marked the first report to the police since deepfake-related provisions were added to the Public Official Election Act.

AI-generated deepfakes are spreading rapidly across election campaigns. As of Wednesday, the NEC had requested the deletion of 8,832 deepfake-related posts connected to the June 3 local elections. Authorities have also issued two criminal complaints, one request for investigation and 32 warnings involving intentional or malicious cases.

The number of posts is increasing by hundreds each day and is expected to surpass the 10,510 cases recorded during last year’s presidential election.

“The NEC will thoroughly prepare for the election with renewed determination,” said NEC Chairperson Roh Tae-ak in a public statement released on Wednesday. “We will respond firmly to acts that undermine election order through the use of deepfake videos.”

Just four years ago, AI-generated videos were viewed less as a regulatory problem and more as an innovative campaign tool.

One of the best-known examples was the “AI Yoon Suk Yeol” video series, featuring an AI-generated avatar of Yoon cheekily answering various submitted questions. The video series was introduced during the 2022 presidential election by Lee Jun-seok, then leader of the People Power Party (PPP) and now the Reform Party leader. The series gained popularity among voters in their 20s and 30s.

But later that June, calls for regulation grew after some PPP candidates used an AI-generated avatar of Yoon to create fake videos that appeared to show the former president endorsing the party candidates' local election campaigns. The NEC had little authority to act because the election law at the time lacked punishment provisions. 

Discussions at the National Assembly’s special committee on political reform that began in November 2023 initially leaned toward a more relaxed regulatory approach: “Let’s broadly allow [AI-generated content], but require it to be clearly labelled as AI-generated.”

Lawmakers even discussed the potential for AI to reduce campaign costs. But momentum shifted after Jang Dong-hyeok, then a member of the committee and the current leader of the PPP, argued that “it is difficult to legally define the scope of fake videos and the potential for abuse is extensive.” His argument helped lead lawmakers toward a blanket ban on such content during the 90 days leading up to an election.

California’s 60-day pre-election deepfake restriction law was referenced during the discussions, while the 90-day standard mirrored an existing rule banning lawmakers from publishing legislative activity reports before general elections.

Despite the revised law, the rapid spread of increasingly sophisticated deepfake content remains a major challenge for election authorities.

Some AI-generated videos can now mimic candidates’ voices almost perfectly or produce videos that are difficult to distinguish with the naked eye.

In response, the NEC has created a 440-member special response team focused on cases involving false information and defamation since December 2025. Authorities have also introduced “Aegis,” a deepfake detection program jointly developed by the National Forensic Service and the Korea Electronics Technology Institute.

“We are increasingly dependent on the detection program because many [AI-generated] videos are difficult to distinguish from real footage with the naked eye,” an NEC official said. “There are also limitations in identifying content shared in closed spaces such as group chats in messenger apps such as KakaoTalk.”

The NEC official added that authorities frequently face complaints asking, “Why are you interfering with posts that were simply made out of fan support?”

Some politicians have also criticized the tougher enforcement approach, arguing that AI can significantly reduce campaign costs and that it is unrealistic to individually police AI-generated content voluntarily created by supporters.

“Standards will likely become more established as enforcement cases and court rulings accumulate through multiple elections,” said Park Myoung-ho, professor at the department of political science and diplomacy at Dongguk University. “It is still difficult to present a clear solution for AI-based election campaigning at this stage.”

This article was originally written in Korean and translated by a bilingual reporter with the help of generative AI tools. It was then edited by a native English-speaking editor. All AI-assisted translations are reviewed and refined by our newsroom.

BY PARK TAE-IN, YEO SUNG-KUK [[email protected]]