Fake images and videos are not a new thing. For as long as photographs and film have existed, people have been fabricating forgeries designed to deceive or entertain, and this has only accelerated since the mass adoption of the internet. But now, rather than images simply being altered by editing software such as Photoshop or videos being deceptively edited, there's a new breed of machine-made fakes – and they could eventually make it impossible for us to tell fact from fiction.

deep fakes are the most prominent form of what’s being called “synthetic media”: images, sound and video that appear to have been created through traditional means but that have, in fact, been constructed by complex software. deep fakes have been around for years and, even though their most common use to date has been transplanting the heads of celebrities onto the bodies of actors in pornographic videos, they have the potential to create convincing footage of any person doing anything, anywhere.

You may have seen a story recently of a popular Twitter account featuring the adventures of a cute female motorcycle enthusiast, who turned out to be a 50-year-old man. Or a woman in the US who faced criminal charges when her daughter’s schoolmate accused her of constructing a deep fake of the girl vaping. The story was national news and the woman was abused on social media, but experts concluded the video was likely genuine and not a deep fake at all. Prosecutors dropped the accusation.

So, in a world already awash with misinformation and untruths, what might be the effect of deep fakes, for individuals and on the political sphere? And, er, how do you know when you're looking at one?

What separates a deep fake from a garden-variety fake?

deep fakes are, in their most common form, videos where one person’s face has been convincingly replaced by a computer-generated face, which often resembles a second person. They came to prominence in 2017 on the online community called r/deep fakes subreddit, whose members began sharing pornographic videos that appeared to feature popular female celebrities. It was clear the celebrities had no part in making the videos, but the techniques used meant they were much more convincing than traditional fakes that simply transposed the face from one video onto the body from another.

The "deep" in deep fake refers to "deep learning". It's this element, and the fact the faces are generated wholly by computer, that makes deep fakes different to — and potentially more dangerous than — fakes that have been manually created by humans.

Deep learning is a method of teaching computers and software that is inspired by the way organic brains learn. It involves the systems processing certain tasks over and over again, sometimes totally unsupervised by humans, in order to learn the best way to turn certain inputs into a desired output. In deep fakes, this means changing one person's face into another's, in ways a human editor might not think of or would be unable to detect.

After the swap, you should be left with a face that has the features specific to Cage, but with the original mannerisms of Adams.

The reason face-swapping is the primary category of deep fake is that all faces share commonalities, even among people of very different appearances, and as long as those elements are consistent our brains tend to believe what they're seeing. If you were imposing Nicolas Cage's face over Amy Adams for example – as in the example above – the software would learn which parts of the original faces were specific to each actor and should be changed (the eye colour, the mouth shape), and which were part of the scene and should be kept (that the eyebrows raise and mouth opens).

After the swap, you should be left with a face that has the features specific to Cage, but with the original mannerisms of Adams.

There's nothing specifically stopping the creation of a deep fake engine that makes it look as though people had guns in their hands when they did not, or makes them look taller than they are, but that doesn't play to deep learning's strengths and would probably be more easily achieved through other means.

It should also be pointed out that, although the technology is very impressive, the idea that a deep fake could literally be indistinguishable from real footage is still only theoretical. Examples such as the one above are clearly fake and incongruous, while an attempt to actually deceive using deep fakes would require incredibly advanced software, meticulous data and a way to imitate the subject’s voice as well as their face.

What kind of deep fake videos are being made?

For the most part, creating deep fakes requires a lot of facial data in the form of pictures and videos, so it's no surprise that almost all of them involve celebrities.

In September 2019, Amsterdam-based company Deeptrace, an organisation "dedicated to researching deep fakes' evolving capabilities and threats", published a study into the almost 15,000 deep fakes circulating online at the time. It found that 96 per cent were pornographic, with 99 per cent of those featuring the faces of female celebrities.

The study also found that the top four websites dedicated to deep fake pornography – the earliest of which was launched in February 2018 – had already attracted almost 135 million video views, highlighting an alarming demand for the non-consensual format.

Advanced techniques promise to produce videos that are more convincing than ever, with even less effort.

Most mainstream sites, including Reddit, have banned deep fake pornography, and several states in the US have already enacted laws outlawing deep fakes involving nudity that have been made without consent. Popular non-pornographic deep fakes involve recasting movies for fun, such as making it appear as though Nicolas Cage plays all the parts in every film.

Since deep fakes typically don't change the voices and sounds of the video, it's also common to alter clips of comedians doing impressions, so they look like the person they're mimicking. Check out this video by impressionist Jim Meskimen and deep fake artist Sham00k:

deep fakes – or similar techniques – have been used in movies. Recent Star Wars films have featured computer-generated versions of Carrie Fisher and Peter Cushing as they appeared in the original 1977 film, while several Marvel movies have “de-aged” actors including Michael Douglas and Robert Downey jnr. Traditional special effects techniques, such as motion capture, help make these much more consistent than your usual internet deep fake, though in some cases less believable.

How difficult is it to make a deep fake?

Typically, creating a convincing deep fake has required a lot of data and a lot of expensive computing power, although advances in technology have meant the techniques are becoming available to a much wider group of content creators than just enthusiasts and professionals. The short answer is, yes, it's difficult – but it might not be in the future.

One of the most popular techniques involves collecting video data of the two people you're swapping and processing it using a very powerful computer you either have physical access to or (more likely) rent using a cloud service. By comparing the different bits of video, the software attempts to learn how to reconstruct the face from all angles.

Many deep fakes result in unconvincing videos where, for example, skin tones come out blotchy or there are clear elements of both people’s faces at the same time. But an experienced faker can account for this by choosing specific people and videos. In the clip below, which is a failed out-take from YouTube channel AsArt, you can see actor Adam Driver’s face hasn’t quite blended with actor Alan Rickman’s in this attempt to deep fake a Harry Potter movie. Notice how the top teeth are missing, and how the software doesn’t appear to have noticed the mismatched forehead.

In another clip from the same channel, this attempt to insert Meryl Streep into the movie has not gone well, potentially because of the moody lighting in the original clip. The software appears confused as to how Streep's face should be coloured.

But advanced techniques promise to produce videos that are more convincing than ever, with even less effort. Special two-part deep-learning systems called generative adversarial networks (GANs) have made headlines for being able to generate anything from original screenplays to paintings of completely invented landscapes. The system essentially plays a game against itself, criticising and evaluating its own output against what it thinks humans will accept as real. In deep fakes, this can make for synthetic videos with no discernible flaws.

Progress has also been made in creating general algorithms for deep fake production. This takes a huge amount of time and computing power but, once complete, an algorithm could potentially allow you to create an instant deep fake video by simply uploading two clips to an app or web browser. Of course, as with viruses and anti-viruses, any technology that can detect or prevent deep fake videos will likely only be temporary, especially since the forgeries are driven by AI that is designed to fool people's perception.

How did deep fakes go 'viral'?

Though many people have expressed concern about politicians being impersonated to fuel misinformation and sway elections, to date the most widely seen, non-pornographic examples have all been celebrity memes.

In August 2019, a deep fake video was uploaded to the YouTube channel Ctrl-Shift-Face of actor Bill Hader on David Letterman's late-night TV show. Here's what he looked like in the original interview, impersonating Tom Cruise:

But in the eye-tricking deep fake clip, Hader's face morphs to resemble the celebrities he's impersonating. Watch what happens as he mimics Cruise:

The clip immediately went viral, for both its subtle creativity and its glimpse into the perception-altering potential of the deep fake form.

In an interview with The Guardian, the video's creator – who identified himself as a Slovakian citizen in the Czech Republic who works as a 3D artist for the film and gaming industries – said he wanted to raise awareness of the subversive potential of deep fakes.

"People need to learn to be more critical. The general public are aware that photos could be Photoshopped, but they have no idea that this could be done with video," he said.

Tom Cruise has since proven a popular subject of deep fakes. There’s a popular TikTok account run by a Belgian visual-effects artist that posts nothing but elaborate fakes featuring the actor.

Actor Steve Buscemi also found himself goofily thrust into debates about the new technology after a clip of his face on Jennifer Lawrence's body – backstage at the Golden Globes, discussing TV show Real Housewives – went viral.

Buscemi's own reaction – "It makes me sad that somebody spent that much time on that," he told late night host Stephen Colbert – echoed the wider online response. As one of the top comments on the video's YouTube page goes: "And everyone worried the internet would use deep fake tech for terrorism."

But could it be used for widespread misinformation?

A common concern is that deep fakes could be used to destabilise democracy or otherwise interfere with politics. It is true that politicians have been the target of many deep fakes. As the US ramped up to its 2020 election, the faces of Donald Trump and Joe Biden proved especially popular.

deep fakes are almost always used to comic effect or in a way where the audience accepts the video is doctored. The technology is not yet at a place where claims of authenticity could be made realistically​. For now, political deep fakes generally take the form of satire, such as in this version of Better Call Saul featuring Donald Trump, which is again from Ctrl-Shift-Face:

However, Jacob Wallis, asenior analyst at the Australian Strategic Policy Institute, says synthetic media doesn't need to hit the worst-case scenario of presidential deep fakes to be a cause for concern. "There are much lower-threshold-level applications in play already that are integrated into influence operations and different disinformation that is currently rippling across social media environments," he says. "Synthetic media covers the full gamut of the kind of media landscape that we engage with when we're online: text, audio, images, video."

For example, it's standard practice for state "actors" and non-state actors to generate synthetic faces using techniques similar to those used in deep fakes, which can be used as profile pictures online to make malicious accounts seem genuine.

Already, just the fact that deepfakes exist has been enough to cause some political trouble.

Meanwhile, foreign actors have also been known to generate synthetic voices using AI, allowing them to add voiceover to videos without giving their accent away.

“We’re seeing these lower level applications of AI-developed content in the public domain, it’s happening now,” Wallis says, adding that those with the power to make convincing video deep fakes right now are likely disinclined to use them.

"Were a state actor to leverage a deep fake of significant quality, sufficient to shift geopolitical events, I think that would lead to significant consequences. So state actors will be thinking cautiously about the thresholds and the deployment of these technologies."

As the technology becomes more democratised, however, non-state actors may not be so reluctant. And, already, just the fact that deep fakes exist has been enough to cause some political trouble.

In Brazil and Malaysia, politicians have distanced themselves from compromising video evidence by claiming they were deep fakes created by opponents.

In a similar case, in 2018, the president of Gabon gave a televised address in response to rumours he had died and the government was covering it up. His political opponents claimed the video was a deep fake, and the military launched a coup. The president turned out to be alive and well.

Outside of the political sphere, a key worry is the use of deep fake technology against regular people, as the ubiquity of video content on social media could create whole new avenues for non-celebrity deep fakes.

There are big implications for cyberbullying if every person has, say, a TikTok account or something similar, full of hundreds of hours of selfie-range video, and if future technology means it's uncomplicated for someone to maliciously turn some of that footage into a deep fake. Celebrities and politicians may be protected by their status and scrutiny, but an ordinary person may have trouble proving their innocence if their peers get hold of video that appears to show them doing something distasteful.