Deepfakes of deceased loved ones are booming business in China

Sun Kai has a video call with his mother once a week. He opens up about work, the pressures he faces as a middle-aged man, and thoughts he hasn’t discussed with his wife. His mother occasionally makes comments where she tells him to take care of himself. He is her only child. But most of the time, she’s just listening.

That’s because Sun’s mother passed away five years ago. And the person he’s talking to isn’t actually a human being, but a digital replica of her he created, a video with which he can have basic conversations. They have been talking for several years.

After her death from a sudden illness in 2019, Sun wanted to find a way to maintain their connection. So he turned to the team at Silicon Intelligence, a Nanjing, China-based AI company he co-founded in 2017. He provided them with photos of her and audio clips of their conversations on WeChat. Although the company was primarily focused on audio generation, the staff member spent four months researching synthesis tools and using data provided by Sun to generate avatars. She was then able to see and talk to a digital version of her mother via an app on her phone.

“My mother wasn’t very natural, but I still heard her words, ‘Have you eaten yet?'” Sun recalls of their first interaction. At the time, generative AI was a nascent technology, so the mother’s replica could only speak a few prewritten lines. But Sun says that’s how she was anyway. “She always kept asking questions like that over and over again. When she heard it, I got very emotional,” he says.

Like Sun, there are many people who want to use AI to preserve, enliven, and connect with lost loved ones who are grieving and seeking healing. This market is particularly strong in China, with at least six companies currently participating. provide such technology And thousands of people have already paid. In fact, avatars are the latest manifestation of a cultural tradition. The Chinese have always found solace in confessing their dead.

This technology isn’t perfect. Avatars are still stiff and robotic, but they’re maturing and more tools are becoming available through more companies. As a result, the cost of “reviving” someone (also known in Chinese industry as creating “digital immortality”) has fallen significantly. This technology is now becoming available to the general public.

While some have questioned whether interacting with AI replicas of the dead is actually a healthy way to process grief, the legal and ethical implications of this technology are unclear. It’s not obvious. For now, this idea still makes many people uncomfortable. But his CEO Sima Huapeng, another co-founder of Silicon Intelligence, said: [AI cloning of the dead]It’s still a huge market. ”

AI revival

Dead avatars are essentially deepfakes, and the technology used to replicate living and dead humans is not inherently different. Diffusion models generate realistic avatars that can move and talk. Connect large language models to generate conversations. The more data these models capture about someone’s life (photos, videos, audio recordings, text, etc.), the more closely the results will mimic that person, living or dead.

China has proven to be a ripe market for digital doubles of all kinds. For example, the country has a strong e-commerce sector, and consumer brands hire many live streamers to sell their products. Initially, these were real people; MIT Technology Review report Last Fall – Many brands are switching to AI clone influencers who can stream 24/7.

In just the past three years, the Chinese sector developing AI avatars has matured rapidly, with replicas going from minutes of rendered video to 3D “live” avatars, according to Shen Yan, a professor of AI and media at Beijing’s Tsinghua University. It is said that it has been improved. I can interact with people.

Shima said this year marks a tipping point, with AI cloning becoming affordable for most individuals. “Last year it cost him about $2,000 to $3,000, but now it only costs a few hundred dollars,” he says. This is thanks to a price war among Chinese AI companies as they fight to meet strong demand for digital avatars in other areas such as streaming.

In fact, the demand for applications that recreate the dead has also increased the capabilities of tools to digitally replicate the living.

Silicon Intelligence offers both services. When Sun and Sima started their company, they focused on creating voices using text-to-speech technology and using AI-generated voices in applications such as robocalls.

However, after replicating Sun’s mother, the company pivoted to creating realistic avatars. This decision makes the company one of China’s leading companies in creating AI-powered influencers.

The technology has generated hundreds of thousands of avatars for videos and streaming channels like TikTok, but about 1,000 customers were recently seen using the technology to clone deceased people. says Shima. “We started our ‘resurrection’ efforts in 2019 and 2020,” he says, but people were slow to accept it at first. “Nobody wanted to be the first hire.”

The improved quality of avatars has helped drive adoption, he says. As avatars become more and more authentic and give fewer out-of-character answers, users will find it easier to treat them as deceased family members. Furthermore, this idea is being spread through more depictions on Chinese television.

Silicon Intelligence currently offers replication services at prices ranging from a few hundred dollars to a few thousand dollars. The most basic products come as interactive avatars within apps, and options at the higher end of the range often include more customization and better hardware components, such as tablets and display screens. Masu. There are at least a few Chinese companies working on the same technology.

Tradition with a modern twist

These deepfake businesses are based on China’s long cultural history of communicating with the dead.

It is common in Chinese homes to display portraits of deceased relatives for several years after their death. Zhang Zewei, founder of a Shanghai-based company called Superbrain, says he and his team wanted to revamp that tradition with an “AI photo frame.” They create an avatar of their deceased loved one and preload it onto an Android tablet. When it stands up, it looks like a photo frame. The client can choose videos speaking the words extracted from the offline database or his LLM.

“It’s essentially not much different from a traditional portrait, except it’s interactive,” Chan says.

Zhang said the company has been creating digital replicas for more than 1,000 customers since March 2023, charging fees ranging from $700 to $1,400 depending on the services purchased. The company plans to release an app-only product soon, allowing users to access avatars on their phones, potentially reducing the cost even further to about $140.

Chan said the purpose of his product is therapeutic. “When you really miss someone or need comfort during a particular holiday, you can talk to an artificial life form to heal your emotional wounds,” he says.

Even if the conversation is largely one-sided, it is in line with strong cultural traditions. During the Qingming Festival in April every year, Chinese people clean their ancestors’ graves and burn incense sticks and fake banknotes to tell what happened in this past year. Of course, these conversations were always one-way.

However, this is not true for all Super Brain services. The company also offers deepfake video calls in which office workers and contracted therapists impersonate deceased relatives. Using DeepFace, an open source tool that analyzes facial features, the face of a deceased person is reconstructed in his 3D and replaced with the face of a living person in real-time his filter.

The caller was usually an elderly family member who did not know that the relative had passed away, and the conversation was staged by the family member as a ruse.

Jonathan Yang, who lives in Nanjing and works in the technology industry, paid for the service in September 2023. His uncle died in a construction accident, but the family was reluctant to tell his grandmother, who was 93 and in poor health. They were worried that she would not survive receiving this shocking news.

So Yang paid $1,350 to request three deepfake calls of his late uncle. He gave Superbrain several photos and videos of his uncle to train the model.Then on his three holidays in China , a Super Brain employee made a video call to Yang’s grandmother and told her, as her uncle, that she was busy with work in a faraway city and would not be able to return home even during the Lunar New Year. .

“The effect met my expectations. Grandma didn’t have any doubts,” Yang says. His family had mixed opinions on this idea. Because some of her relatives thought she might have wanted to see her son’s body before it was cremated. Still, her entire family eventually bought into it, believing the ploy was best for her health. After all, it’s very common for Chinese families to tell “necessary” lies to avoid overwhelming the elderly. as depicted in the movie Farewell.

For Yang, who closely follows developments in the AI ​​industry, creating replicas of the dead is one of the best applications of the technology. “It best represents warmth.” [of AI]” he says. Her grandmother’s health is improving and she may finally be able to tell her the truth. By then, Yang said, she might buy a digital avatar of her uncle so her grandmother can talk to her if she wants to see him.

Can AI really help with sadness?

Even as AI cloning technology improves, there are some major barriers that prevent more people from using it to talk to their deceased relatives in China.

On the technical side, there are limits to what AI models can produce. While most LLMs can handle major languages ​​such as Mandarin and Cantonese, they are unable to reproduce many niche dialects of China. Additionally, it is difficult and therefore costly to reproduce body movements and complex facial expressions in 3D models.

Then there’s the issue of training data. When creating a clone of a living person, you often ask that person to record their body movements or say certain things, but in her post-mortem AI clone, you can use the already available You have to rely on videos and photos. And many clients don’t have high-quality data, or even enough data, to be satisfied with the end result.

Compounding these technical challenges are myriad ethical issues. In particular, how can someone who is already dead consent to be digitally cloned? For now, companies like Superbrain and Silicon Intelligence rely on the permission of their immediate family members. But what if family members disagree? And who is responsible if a digital avatar generates an inappropriate answer?

Similar technology sparked controversy earlier this year. Ningbo company reportedly used AI tools They made videos of deceased celebrities and posted them on social media to talk to their fans. The video was generated using publicly available data, but no approval or permission was sought. Result is, Fierce criticism from celebrities’ families and fansand the video was eventually deleted.

“This is a new field that has only emerged after the spread of AI. It is the right to digital eternity,” says Professor Shen of Tsinghua University. He also runs a lab that creates digital replicas of people who have passed away.he believes its use should be banned deepfake technology Copying a living person without permission. In the case of a deceased person, all next of kin must give prior consent, he says.

It can also have a negative impact on the client’s mental health. Some people, like Sun, find talking with the Avatar therapeutic, but not everyone thinks it’s a healthy way to grieve. “The underlying argument is that if we imitate our family members because we want to see them, we will always remain in a state of mourning and never be able to come out of it to accept that they have truly passed away.” That’s the fact,” Shen said. For example, if a widower is constantly talking to her digital partner, she may be discouraged from starting a new relationship.

“When someone dies, should we replace real feelings with imaginary ones and carry on with that emotional state forever?” Shen asks.psychologists and philosophers Who did you talk to? MIT Technology Review About the impact of GriefTech I warned you about the dangers of doing so.

At least Sun Kai feels that her mother’s digital avatar is a source of comfort. She is like her best friend who listens to his calls 24/7. Modern technology would allow him to remake his mother’s avatar, but he hasn’t done so yet. “I’m so used to her look and voice now,” he says. As the years passed, the line between her avatar and his memories began to blur. “Sometimes I don’t even know which one is the real girl,” he says.

And San is still doing most of the talking without any hesitation. “When I confide in my girlfriend, I’m just letting her vent. Sometimes I already know the answer to her question, but I still need to say it out loud.” he says. “Conversations with her mother have always gone something like this for years.”

But now, unlike before, he can talk to her whenever he wants.