By Shane Raymond

(Reuters) – “Do you wish to be seen in a film or on television?” The description of an app available on the internet, that allows users to create artificial media using AI, is called deepfakes.

Do you wish to watch your boss, friend or colleague dance? It added. “Have You ever wondered what your face would look like if it were swapped with that of a celebrity or your friend?” 

On dozens of adult websites, the same app was promoted differently: The ads claimed that you could “make deepfake porn within a second.” “Deepfake anyone.”

The complexity of synthetic media software is due to the increasing sophistication with which technology is applied. Machine learning can be used to digitally create faces from images, and then seamlessly convert them into films.

According to Reuters interviews, the technology is only four years old and may have reached a critical point.

Experts said that the technology is now so advanced that viewers will struggle to tell the difference between fake and real videos.

Adam Dodge (an attorney and founder of EndTab online safety platform) stated, “Once you have a low entry point, it doesn’t require any effort, and an unsophisticated user can create very sophisticated non-consensual pornographic videos – that is the inflection point.”

“This is where you get into trouble.”

Many online safety advocates, researchers, and software developers believe that consent is the most important thing to do when simulating someone’s life. However, this can be difficult. Given the potential abuse, some advocate a more aggressive approach to synthetic pornography.

Sensity, which monitors fake media and detects it, reported that 96% of the sample of over 14,000 videos online contained non-consensual deepfake content. The report stated that deepfake video online had been increasing by approximately doubling in six months.

Ajder was one of the study’s authors, as well as the head for policy and partnerships at AI firm Metaphysic. He said, “The vastly vast majority” of harm done by deepfakes is caused by gendered online violence.

According to him, it is important that an app not explicitly market itself as a pornographic device is clearly marked.

AD NETWORK XES APP

ExoClick was the online advertising platform that was used in the app “Make deepfake Porn in a Sec” and told Reuters it wasn’t familiar with the AI face-swapping technology. It stated it had stopped the app taking out ads and wouldn’t promote face-swap technology irresponsibly.

Bryan McDonald, ExoClick’s chief of ad compliance, said that this product type is brand new. ExoClick offers clients, like large networks, access to a variety of websites they can customize to determine where to place ads.

We have reviewed the marketing material and determined the words used are unacceptable. While we are certain that most users use these apps for entertainment, it is possible for them to be misused for malicious purposes.

Six other major online ad network were not contacted by Reuters to comment on requests for information about deepfake software they’ve encountered or a policy.

The description of the app on Apple (NASDAQ) or Google (NASDAQ) does not mention its possible pornographic use. It is also available for anyone who is over 12 years old.

Apple stated that deepfake apps are not covered by its rules, however, it did say that the guidelines prohibit any apps which contain content that could be defamatory, discriminatory, or likely to intimidate, humiliate, or harm others.

The company stated that the App Store prohibited developers from misleading customers and was working closely with their development company to make sure they adhered to its guidelines.

Google has not responded to any requests for comment. After being contacted Reuters by concern about “Deepfake Porn” advertisements on adult websites, Google removed the Play Store page of the app’s Play Store, which was rated E. After two weeks of restoration, the page was rebuilt with T-rated Teen content.

WATERMARKS AND FILTERS

Although there are many bad actors in the rapidly growing face-swapping app industry, there is a large variety of apps that consumers have access to and they do take steps towards preventing abuse. Ajder who advocates the ethical use synthetic media within the Synthetic Futures group said,

Some apps don’t allow for users to share images with others.

Reface has been downloaded more than 100,000,000 times worldwide since 2019. Users are encouraged to swap faces with famous people, heroes, and meme characters in order to make fun videos.

According to Reuters in the U.S., the company said that it uses both human and automatic moderation to control content. This includes a pornography filters and other controls to protect against misuse.

It stated that “from the inception of Reface’s technology to the establishment of Reface, we have been aware of the possibility of synthetic media technology being misused or abused.”

“ONLY PERPETRATOR LIABLE”

With the increasing consumer access to high-powered computing through smartphones, comes advances in deepfake tech and quality synthetic media.

EndTab founder Dodge, along with other experts interview by Reuters, stated that they needed a lot of data input in 2017 to get the same quality as today. This included thousands of images.

With the image quality increasing, it is becoming increasingly difficult to protest “It’s NOT me!” They aren’t enough and it doesn’t matter if the images look like you.

Deepfake technology regulation is proving difficult for policymakers, who are also facing new ethical and technical challenges.

In some countries, laws have been passed that specifically target online abuse of deepfake technology. This includes California, South Korea, China and South Korea. A law that maliciously depicts someone in pornography or distributes such material without their consent can result in statutory damages up to $150,000.

Researchers at the European Parliament stated that “Specific legislative intervention is not yet possible to criminalize deepfake pornography,” in an October study to a group of legislators. They suggested that legislation could include distributors or developers as well as those who abuse them.

According to current law, only the perpetrator can be held liable. But many of these perpetrators will go to extreme lengths in order to launch such attacks anonymously so that neither platforms or law enforcement can identify them.

Marietje Schaake (international policy director, Stanford University’s Cyber Policy Center) said that new broad digital laws could include the AI Act in America and GDPR for Europe. However, there were still some gaps.

Schaake explained that although it might sound like there are many legal avenues to choose from, the reality is that victims must be given the power to take these actions.

She added that “The AI Act draft under review foresees that the manipulated content should also be disclosed.”

The question here is whether awareness alone can stop this harmful effect. The virality of conspiracy theories shows that even information too unbelievable to be true could still have a wide-reaching and damaging social impact.