10 months ago
75 ARTICLE AD
Bengaluru, India – On January 23, an icon of Indian cinema and politics, M Karunanidhi appeared before a live audience on a large projected screen, to congratulate his 82-year-old friend and fellow politician TR Baalu on the launch of his autobiographical book.
Dressed in his trademark black sunglasses, white shirt, and a yellow shawl around his shoulders — Karunanidhi’s style was spot on. In his eight-minute speech, the veteran poet-turned-politician congratulated the book’s author but was also effusive in his praise for the able leadership of MK Stalin, his son and the current leader of the state.
Karunanidhi has been dead since 2018. This was the third time, in the past six months, that the iconic leader of the Dravida Munnetra Kazhagam (DMK) party was resurrected using artificial intelligence (AI) for such public events.
“When the COVID pandemic ravaged the world, our Chief Minister ran in the direction of panicked voices of people,” Karunanidhi said. “The nation knows the way you fought to save the lives of people, and so do I.”
Senthil Nayagam, founder of Muonium, the AI media tech firm that made the deepfake Karunanidhi video, told Al Jazeera that “there is a market opening up [for such deepfakes]…. You can attribute some statements to a particular person and that kind of gives more value to it”.
AI Karunanidhi’s first public appearance was at a local media event last year in September, which was followed up by another for a campaign by his party members. The resurrected leader often felicitates party workers and specifically praises the leadership of his son MK Stalin — with the aim of boosting his popularity.
At the January book launch, AI Karunanidhi recounted everything from pardoning student debt and cash giveaways for the poor, to female-friendly policies and roping in investments — a list of his son’s achievements over the years that had propelled the state forward.
Karunanidhi’s last public interview was in 2016, before his voice turned coarse, and his body frail. Nayagam used publicly available data of Karunanidhi to train a speech model and recreated the 1990s likeness of the leader when he was much younger. The script for the prerecorded AI speech, he said, was supplied by the local DMK cadre, and was vetted by party personnel.
TR Baalu, whose team sanctioned the creation of AI Karunanidhi, did not respond to Al Jazeera’s request for comment.
Karunanidhi was one of India’s longest-serving legislators who helmed the state of Tamil Nadu for nearly two decades, serving a total of five terms as chief minister. The poet-turned-politician wrote screenplays about lower caste emancipation and continues to hold sway among older voters.
As per local media reports, the reaction to these AI videos has prompted the DMK party leadership to think of creating AI Karunanidhi campaign speeches in the upcoming 2024 parliamentary election campaign.
An ethical and legal quandary
Even as policymakers evaluate instances of the types of AI communication that should be regulated, in a first-of-its-kind use, a political party used AI to resurrect a yesteryear political stalwart to promote today’s leader.
But it has also raised some troubling ethical and legal questions: “The use of AI to create synthetic audio and video by a living person who has signed off on the content is one thing. It is quite another to resurrect a dead person and ascribe opinions to them,” said Amber Sinha, senior fellow for Trustworthy AI at Mozilla Foundation.
But the genie is already out of the bottle. According to Diggaj Mogra, director of Jarvis Consulting, one of India’s largest political consultancies, AI-facilitated content marketing for elections campaigns, including outbound voice calls and SMS, avatar creation, personalised media outreach, and AI-created multilingual creatives on social media is an estimated $60m market opportunity in India this election year.
“In Tamil Nadu, all the big leaders of each party are no more,” Nayagam said, referring to former actors-turned-politicians Jayalalitha, MG Ramachandran, and Vijayakanth. Nayagam said he has been in touch with several low-level functionaries across party lines interested in leveraging AI for similar deepfakes.
Interest in such applications ballooned, he said, after he shared last year on X in September a four-minute audio clip of Prime Minister Narendra Modi’s Mann Ki Baat program that his firm had cloned in eight languages. Those inquiries of interest have given Nayagam and other consultants the idea of a business opportunity in AI electioneering.
Globally, over 60 countries are set to hold national elections in 2024, and the possible misuse of artificial intelligence to influence public opinion has caused a moral panic, turning into a global hot-button issue.
Ahead of the Indonesian presidential campaign, Prabowo Subianto, a former military general accused of committing atrocities against pro-democracy activists, is using generative AI to reimagine himself as a chubby-cheeked AI avatar, to attract young voters.
In South Asia, AI’s use for campaigning and instances of misuse has gained prominence. In Bangladesh, pro-government accounts have used deepfakes to target opposition parties. In Pakistan, former Prime Minister Imran Khan has been campaigning from inside his prison cell by passing on written notes to his lawyers, which are being turned into AI audio speeches using software from US-based start-up ElevenLabs.
“This particular use of AI in campaigns seems to be taking off in South Asia in a big way,” said Sinha of Mozilla Foundation.
On January 21, the DMK party organised its second annual youth wing conference in the temple town of Salem. The mega event hosted in an open arena drew a crowd of 500,000 supporters and marked the official launch of the 2024 election campaign of the DMK. Party leaders offered rousing speeches challenging the Hindu nationalist Bharatiya Janata Party (BJP), and criticised its policies including the dilution of the powers of the states by the BJP-ruled centre.
At this campaign, AI Karunanidhi made a surprise video reappearance. “Many hard-fought states’ rights have been lost in the 10-year BJP rule,” Karunanidhi said, elaborating on the continued hostility of the BJP towards Tamil Nadu.
The three-minute video speech, accompanied by inspiring music, concludes with AI Karunanidhi calling for the strengthening of the state’s rights, and urging young cadres to fight for a democratic future.
Screengrab of AI Karunanidhi at DMK’s second annual youth wing conference
“This was created by the party’s digital media wing to encourage and enthuse the cadres,” DMK spokesperson Dharanidharan Selvam told Al Jazeera. “I think the cadres were definitely enthusiastic and excited.”
Deceased leaders are in vogue for political campaigns “because they continue to be more popular than the living ones”, said Sumanth Raman, a Chennai-based political commentator. “You don’t have mass leaders of the calibre of Ms Jayalalitha” — another political starlet — “or Mr Karunanidhi in Tamil Nadu at the moment”.
In the past 30 years, over half a dozen political parties have been founded by actors in the state. Many leaders came from the world of cinema where they played larger-than-life characters, Raman said, and were always put on a pedestal by the people.
“But that’s sort of tailored off when we move to the next generation, which is where we are now. Leaders face much greater scrutiny on a day-to-day basis, and therefore you don’t have this huge aura about that,” he said.
‘Leverage the popularity of a dead person’
The impact on audiences of resurrecting iconic leaders is still unfolding. “I think that was pretty ordinary as an effort — it went to their party men so they lapped it up,” said Raman, about the AI video speech. “Today’s AI is capable of even better imagery.”
For the book launch video, it is clear the video is synthetic, as the lip sync does not quite match. Yet, the audio of Karunanidhi’s voice mirrors reality.
Nayagam, the creator of AI Karunanidhi said, one of the reasons for the imperfect visuals was the unavailability of high-quality video datasets, forcing them to source whatever was available on the internet.
The video aired at the youth conference was better, but was inconsistent near the mouth. Still, the online audience response to the videos was favourable, with some commenting “super” on YouTube. Both videos were clearly labelled as AI-generated.
India witnessed the first-ever use of deepfakes in election campaigning in 2020, when BJP politician Manoj Tiwari sanctioned the creation and distribution of deepfake videos of himself campaigning in Haryanvi and English, languages he does not speak. Experts decried the video, but on the grounds that it was shared without disclosure that it is AI-manipulated.
One may argue that creating AI videos of politicians is an extension of the use of photographs or images of dead persons by their political parties, Sinha said, such as the use of images of Nehru, Indira Gandhi or Rajiv Gandhi after their deaths by the Indian National Congress.
“However, creating synthetic audio or video goes several steps further,” Sinha said. “In both cases, the party is trying to leverage the popularity of a dead person, but in the latter, opinions and messages are actively ascribed to them.”
An additional conundrum in resurrecting a deceased politician is: Who owns the rights to the dead person’s voice and likeness?
“This, of course, doesn’t have legal grounding in India, because we don’t have any enshrined rights of deceased people in India, but from an ethical point of view, consent needs to be considered,” said Devika Malik, a Delhi-based tech policy consultant focused on online trust and safety. Indian law offers legal protection against defamation of a dead person.
Indian politicians are also actively seeking AI solutions for upcoming campaigns. Mogra of Jarvis Consulting said individual candidates are trying to use AI voice clones to push outbound robocalls or IVRS (Interactive Voice Response System) with recorded voice and personalised names in their messages. “There are multiple vendors running around cross country to do this, selling this at very nominal rates,” Mogra said.
There already existed a burgeoning synthetic media economy on the sanctioned use of deepfakes by actors and CEOs, and that market has expanded into the political realm. Consultancies such as Polymath Solutions, operated by Devendra Singh Jadoun, are using voice cloning to deliver “personalised messages” to on-ground party workers from politicians.
“It will be used massively in this election,” Mogra said. However, “it’s a double-edged sword. It will create a lot of misinformation and disinformation. I think benefits are less, and mis/disinformation issues and all will be higher.”
From a social impact standpoint, to what extent these AI videos — even poorly made ones — could shape voter attitudes remains unclear.
Even when such videos are not high quality, “in the case of a popular past speaker such as M Karunanidhi, it can lead to more eyeballs for the messaging and help it go viral”, Sinha said.
His latest research highlights how “diffuse actors” or political consultants, who despite no party affiliation, collaborate with campaigns to spread their message.
Leveraging sentimental appeals from specific stalwarts or families — especially when it is personalised and sent on WhatsApp — can be an effective communication strategy and could sway voter opinion, Malik added.
But perspectives of policy advocates and practitioners sharply diverge on synthetic media’s effectiveness.
Jarvis’ Mogra predicts that the novelty factor of AI-based personalised messages by politicians — be it audio or video — would wear out soon. “If people start seeing it a lot, they will realise that it’s happening everywhere, and they are seeing it everywhere — just like what has happened with WhatsApp,” he said.
Previously, political parties exhibited extra emphasis on creating WhatsApp groups for outreach. Now, everyone knows that each person is part of hundreds of groups and no one pays much attention or reads all the chats.
“I think we will face similar issues at a much faster speed with these generative AI solutions and use cases,” Mogra said.
