AI deepfakes by MyHeritage bring your ancestors to life

You can bring your forebears to life in videos with the help of artificial intelligence. But should you?

Using MyHeritage.com's deepfake video capability

You can bring your forebears to life with the help of artificial intelligence. Instead of viewing a static old black-and-white photo taken a hundred years ago, you can watch a video of them looking around, tilting their head, looking pensive and smiling.

Is this a good thing? I played around with this feature and asked my family what they thought. Some found it unsettling and uncomfortable seeing someone seemingly brought back to life. This was especially when it was an aged relative they knew in their childhood, a fading memory uncomfortably jolted.

Others saw it as seeing that person before them as in everyday life today. They looked like a real person, rather than an artefact from the past.

Having old photos in .MP4 video format means you can transform an album of your forebears into a movie reel, a video album reel that can be shared on the internet. You could add photos of old homes and turn it into a family documentary.

This idea is another example of deepfakes and uses machine learning to understand the parts of a person’s face and then merges those parts onto movements created by the technology developers.

In other applications you can add sound and even voice recreated from original audio.

Deepfakes have garnered a bad reputation. There are deepfakes of politicians and celebrities saying things they never said, fake news based on deepfake interviews, financial frauds and hoaxes.

Online you can create your own deepfake videos of you looking younger or older.

Bringing your ancestors to life is a marketing project of MyHeritage.com, one of several cloud-based sites specialising in tracing your ancestry and building family trees. The company compiled a deepfake video with a reanimated Abraham Lincoln advertising its services.

You upload a still photo with one or more faces to MyHeritage’s Deep Nostalgia page, and it creates an MP4 video to download.

It works better with photos with largish images of faces, good resolution and contrast. Photoshop or Pixelmator can help to a point. However the technology can pinpoint the faces in a photo and create video around each one. In photos with multiple faces, the results are separate videos of each person.

MyHeritage has licensed the technology from Israeli artificial intelligence firm D-ID, which originally specialised in building face recognition technologies that “de-identify” images by removing key biometric data. The person may look the same but the electronic image no longer can be used for surveillance.

MyHeritage has adapted this technology to create its “deep nostalgia” feature. It’s like having Apple Live Photos or Google Motion Photos applied to very old photos a hundred years or more old.

MyHeritage said more than one million photos were animated in the first 48 hours when the feature debuted a week ago, with three million photos now coming in a day.

“Some were awed to see ancestors they’d never met — some from over 100 years ago — move, blink, and smile, while others were moved to tears witnessing their lost loved ones in motion after so many years with only still photos to remember them by,” MyHeritage said.

I used Deep Nostalgia to create a video of four family members. It’s a start of a larger family reel I could create over time. I suspect people will see it either as a warm tribute to people long gone, or macabre and unsettling.

While the results are astounding, I have a few observations. First, the mannerisms, eyes and movements are not those of loving relatives who may have had different mannerisms. How would MyHeritage know? In fact, how would I have known the way they moved before I was born?

Second, the side of a face in the video won’t be accurate if the photo doesn’t show it. In my case, it seemed to do a great job nonetheless.

Third, the eyes can look different. I saw how small changes in the shape of an eye can heavily impact a person’s appearance. The video is displayed above.

To build in the movement, MyHeritage said it produced “driver videos” of people moving their heads about, and its program seeks to apply the most appropriate recorded movements to an uploaded photo.

“The gestures in the driver videos that are used to create the animation sequences are real human gestures and the actors in most of these blueprint videos are employees of MyHeritage,” it says on its website.

“However, the end result is not authentic — it’s a technological simulation of how the person in your photo would have moved and looked if they were captured on video.”

The Deep Nostalgia service is initially free but you’ll need a complete MyHeritage subscription for continuing access – it’s on special at $19.92 per month but usually $29.08 monthly. That’s a lot if your interest is just deepfakes, not the site‘s broader services.

MyHeritage Deep Nostalgia

The real work was done by D-ID. Speaking with the Australian from Israel, D-ID chief executive Gil Perry said the 4-year-old company had embarked on new deepfake learning projects during the pandemic.

Currently they are filming an ad with a high profile sport’s personality who can’t travel to the set location. The company is artificially creating the sport’s stars presence at the location using deepfake technology. “It’s happening right now,” he told The Australian.

He said the company began by building technology to protect people’s images and video from face recognition (de-identification). For example, with D-ID technology, a photo submitted to authorities for your driver’s licence couldn’t be used as a means of later identifying you using government cameras.

Founders of Israeli deepfake specialist firm D-ID (left to right) Eliran Kuta (CTO), Gil Perry (CEO) and Sella Blondheim (COO).

He said the MyHeritage project was just a small part of what D-ID deepfake technology could achieve; he was adamant the firm’s technology must be used for “good intentions”.

He said that in future, a service like MyHeritage might not only clone faces, but also clone body movements so that a customer could recreate a family member from long ago walking down the road, or holding a conversation. “Yes, body and voice cloning is on the road map.

“We’re pretty advanced in body, voice and scene,” he said.