Experts worry that voters in Pakistan, India, and other Asian countries could be duped by AI deepfakes

While producing voice clones and artificial intelligence-based visual effects for Indian cinema and television, Divyendra Singh Jadoun started receiving calls from politicians asking whether he could produce AI films, or deepfakes, for their campaign.

His startup, The Indian Deepfaker, has a great chance ahead of it with a national election scheduled for May of this year and a fiercely contested local election in his home state of Rajasthan last November. Jadoun, though, was hesitant.

“The technology to create deepfakes is so good now, it can be done almost instantaneously, with very little effort – and people cannot tell if it’s real or fake,” Jadoun, 30, said.

“Deepfakes lack regulations, which is concerning because it could affect an individual’s voting behavior,” he told the Thomson Reuters Foundation.

Recent popular content includes Instagram reels with Indian Prime Minister Narendra Modi singing in regional languages and TikTok videos featuring Prabowo Subianto and Anies Baswedan, two Indonesian presidential contenders, speaking Arabic well.

However, they were all made by AI and shared without a label.

Misinformation is common on social media platforms ahead of the upcoming elections in Bangladesh, India, Indonesia, and Pakistan. Tech experts and officials say deepfakes, which are videos or audio produced using artificial intelligence and broadcast as real, are especially worrying.

Deepfake videos are a “big concern” for Indian Prime Minister Narendra Modi, who is among the country’s over 900 million eligible voters. Authorities have alerted social media platforms that failing to take action could result in the loss of their safe-harbour status, which shields them from liability for content posted by third parties on their platforms.

Deepfakes of all three presidential candidates and their running mates are circulating online in Indonesia, where more than 200 million people are expected to cast ballots on February 14. These fakes have the ability to sway election results, according to Nuurrianti Jalli, a social media misinformation specialist.

“From microtargeting of voters with disinformation to spreading false narratives at a scale and speed unachievable by human actors alone, these AI tools can significantly influence voter perceptions and behaviour,” she stated.

“AI-generated content can further skew public perception and influence voting behavior in environments where misinformation is already prevalent,” said Jalli, an assistant professor at Oklahoma State University’s media department.

There have been increasing worries about the impact of deepfake images and videos produced by generative AI tools like Midjourney, Stable Diffusion, and OpenAI’s Dall-E on US presidential polls in November after they appeared before of elections in Turkey, Argentina, and New Zealand last year.

AI speeds up, lowers the cost, and increases the effectiveness of misinformation production and dissemination, according to a new analysis from US nonprofit Freedom House.

Following polls on January 7, Prime Minister Sheikh Hasina of Bangladesh was declared to be serving a fourth consecutive term. However, films featuring female opposition politicians Rumin Farhana in a bikini and Nipun Roy in a swimming pool have surfaced, which are deepfakes.

Even though they were swiftly disproved, deepfakes continue to be shared and mislead people, according to Sayeed Al-Zaman, an assistant professor of journalism at Bangladesh’s Jahangirnagar University who specializes in social media studies.

“Given the low levels of information and digital literacy in Bangladesh, deepfakes can be potent carriers of political propaganda if crafted and deployed effectively,” he stated.

“But the government does not appear concerned.”

An inquiry for comment was not answered by the ministry of information.

Imran Khan, who was removed from office as prime minister last year and is currently serving a prison sentence for violating the Official Secrets Act, addressed an online election rally in December that attracted tens of thousands of live viewers and received over 1.4 million views on YouTube. The election is set to take place in Pakistan on February 8.

Although Pakistan has proposed an AI law, advocates for digital rights have criticized the absence of safeguards against misinformation and to protect women and other vulnerable areas.

“It is imperative to emphasize the danger that misinformation presents to Pakistan’s democratic process and election system,” stated Nighat Dad, a co-founder of the nonprofit Digital Rights Foundation.

“In the past, misinformation distributed online has been able to affect party support, voting patterns, and even legislative changes. This will be made simpler by synthetic media,” she continued.

DeepMedia, a business that creates tools to identify synthetic media, estimates that in 2023, at least 500,000 video and voice deepfakes were shared globally on social media platforms.

Platforms have had difficulty keeping up.

Facebook, Instagram, and WhatsApp parent company Meta stated that it seeks to eliminate artificial intelligence-generated content when it “may mislead, especially when it comes to video content.”

YouTube’s parent company, Google, stated in November that in order to use the platform, “creators must disclose altered or synthetic content that is realistic, including using AI tools, and we’ll inform viewers about such content through labels”.

Raman Jit Singh Chima, Asia policy director at advocacy group Access Now, claimed that platforms are “holding their punches” because nations like Bangladesh, Indonesia, and India have recently passed laws to more strictly regulate online content and penalize social media sites for content deemed misinformation.

According to these nations, “this election cycle is actually worse than the last one because platforms are not responsive and proactive enough, nor are they set up to handle problems.” And that’s a pretty bad indication,” he remarked.

“There is a danger that the world’s attention is only on the US election, but the standards being applied there, the effort being made there should be duplicated everywhere,” he stated.

Jadoun, who had declined to create deepfake campaign films for the state polls, is preparing to do so for the general election in India, where Modi is expected to win a third term.

These will be customized WhatsApp video messages from politicians to party members, not to voters.

“They can really have an impact, because there are hundreds of thousands of party workers and they will, in turn, forward them to their friends and family,” he stated.

To avoid any confusion, we will include a watermark indicating that artificial intelligence was used in its creation. That is crucial.