Deepfake pornography
Explicit material applying deepfake technology
- 12 min read
- 4 chapters
Related on Lably
Through Consent, Outline of machine learning and Artificial intelligence
History
The term "deepfake" was coined in 2017 on a Reddit forum where users shared altered pornographic videos created using machine learning algorithms. It is a combination of the word "deep learning", which refers to the program used to create the videos, and "fake" meaning the videos are not real. Deepfake pornography is a type of fake nude photography.
Deepfake pornography was originally created on a small individual scale using a combination of machine learning algorithms, computer vision techniques, and AI software. The process began by gathering a large amount of source material (including both images and videos) of a person's face, and then using a deep learning model to train a Generative Adversarial Network to create a fake video that convincingly swaps the face of the source material onto the body of a pornographic performer. However, the production process has significantly evolved since 2018, with the advent of several public apps that have largely automated the process. While several AI "nudification" apps emerged on mainstream platforms like Google Play and the Apple App Store around 2023, major tech storefronts have since implemented stricter policies and automated detection to ban such software. Consequently, the proliferation of non-consensual deepfake pornography has largely shifted to decentralized websites, specialized online forums, and third-party messaging bot ecosystems.
Sign in to keep reading
3 more chapters are free with a Lably account.
Notable cases
Deepfake technology has been used to create non-consensual and pornographic images and videos of famous women. One of the earliest examples occurred in 2017 when a deepfake pornographic video of Gal Gadot was created by a Reddit user and quickly spread online. Since then, there have been numerous instances of similar deepfake content targeting other female celebrities, such as Emma Watson, Natalie Portman, and Scarlett Johansson. Johansson spoke publicly on the issue in December 2018, condemning the practice but also refusing legal action because she views the harassment as inevitable.
Rana Ayyub
In 2018, Rana Ayyub, an Indian investigative journalist, was the target of an online hate campaign stemming from her condemnation of the Indian government, specifically her speaking out against the rape of an eight-year-old Kashmiri girl. Ayyub was bombarded with rape and death threats, and had a doctored pornographic video of her circulated online. In a Huffington Post article, Ayyub discussed the long-lasting psychological and social effects this experience has had on her. She explained that she continued to struggle with her mental health and how the images and videos continued to resurface whenever she took a high-profile case.
Atrioc controversy
In 2023, Twitch streamer Atrioc stirred controversy when he accidentally revealed deepfake pornographic material featuring female Twitch streamers while on live. The influencer has since admitted to paying for AI generated porn, and apologized to the women and his fans.
Schools in Spain and New Jersey
In a town in southern Spain in 2023, boys made deepfake pornography, using the website ClothOff, that depicted at least 20 local girls between 11 and 17 years old. After a police investigation, the boys received suspended sentences for distribution of child sexual abuse material. In New Jersey in 2023, students at Westfield High School used ClothOff to generate explicit deepfakes of classmates who were underage girls, based on ordinary photos downloaded from social media. One of the girls coordinated with a Yale Law School professor to sue ClothOff in October 2025.
Taylor Swift
In January 2024, AI-generated sexually explicit images of American singer Taylor Swift were posted on X (formerly Twitter), and spread to other platforms such as Facebook, Reddit and Instagram. One tweet with the images was viewed over 45 million times before being removed. A report from 404 Media found that the images appeared to have originated from a Telegram group, whose members used tools such as Microsoft Designer to generate the images, using misspellings and keyword hacks to work around Designer's content filters. After the material was posted, Swift's fans posted concert footage and images to bury the deepfake images, and reported the accounts posting the deepfakes. Searches for Swift's name were temporarily disabled on X, returning an error message instead. Graphika, a disinformation research firm, traced the creation of the images back to a 4chan community.
A source close to Swift told the Daily Mail that she would be considering legal action, saying, "Whether or not legal action will be taken is being decided, but there is one thing that is clear: These fake AI-generated images are abusive, offensive, exploitative, and done without Taylor's consent and/or knowledge."
The controversy drew condemnation from White House Press Secretary Karine Jean-Pierre, Microsoft CEO Satya Nadella, the Rape, Abuse & Incest National Network, and SAG-AFTRA. Several US politicians called for federal legislation against deepfake pornography. Later in the month, US senators Dick Durbin, Lindsey Graham, Amy Klobuchar and Josh Hawley introduced a bipartisan bill that would allow victims to sue individuals who produced or possessed "digital forgeries" with intent to distribute, or those who received the material knowing it was made non-consensually.
2024 Telegram deepfake scandal
It emerged in South Korea in August 2024, that many teachers and female students were victims of deepfake images created by users who utilized AI technology. Journalist Ko Narin of The Hankyoreh uncovered the deepfake images through Telegram chats. On Telegram, group chats were created specifically for image-based sexual abuse of women, including middle and high school students, teachers, and even family members. Women with photos on social media platforms like KakaoTalk, Instagram, and Facebook are often targeted as well. Perpetrators use AI bots to generate fake images, which are then sold or widely shared, along with the victims' social media accounts, phone numbers, and KakaoTalk usernames. One Telegram group reportedly drew around 220,000 members, according to a Guardian report.
Investigations revealed numerous chat groups on Telegram where users, mainly teenagers, create and share explicit deepfake images of classmates and teachers. The issue came in the wake of a troubling history of digital sex crimes, notably the notorious Nth Room case in 2019. The Korean Teachers Union estimated that more than 200 schools had been affected by these incidents. Activists called for a "national emergency" declaration to address the problem. South Korean police reported over 800 deepfake sex crime cases by the end of September 2024, a stark rise from just 156 cases in 2021, with most victims and offenders being teenagers.
On September 21, 6,000 people gathered at Marronnier Park in northeastern Seoul to demand stronger legal action against deepfake crimes targeting women. On September 26, following widespread outrage over the Telegram scandal, South Korean lawmakers passed a bill criminalizing the possession or viewing of sexually explicit deepfake images and videos, imposing penalties that include prison terms and fines. Under the new law, those caught buying, saving, or watching such material could face up to three years in prison or fines up to 30 million won ($22,600). At the time the bill was proposed, creating sexually explicit deepfakes for distribution carried a maximum penalty of five years, but the new legislation would increase this to seven years, regardless of intent.
By October 2024, it was estimated that "nudify" deep fake bots on Telegram were up to four million monthly users.
2025–2026 Grok/X chatbot deepfake scandal
In December 2025, Bloomberg reported that X users found Grok would comply with non-consensual requests to digitally undress individuals, including minors, or show them performing sexually explicit acts. The majority of these prompts were targeted at women and girls. An analysis of 20,000 images generated by Grok between December 25, 2025 and January 1, 2026 showed 2% were of people in bikinis or transparent clothes and appeared to be 18 or younger, including 30 of "young or very young" women or girls. A separate analysis conducted over 24 hours from January 5 to 6 calculated that users had Grok create 6,700 sexually suggestive or nudified images per hour. xAI responded to requests for comment from media organizations with the automated reply, "Legacy Media Lies". The bot's image generation sparked an international backlash and calls for legal or regulatory action from officials in the European Union, United Kingdom, Poland, France, India, Malaysia, and Brazil.
Fernandes–Ulmen case
In March 2026, German TV presenter Collien Fernandes filed a complaint against her ex-husband, actor Christian Ulmen, for several accusations including identity theft, public defamation, and assault. She alleged that over several years, he created multiple false social media accounts of her, distributing fake nude photos and videos of her.
Ethical considerations
Deepfake child pornography
Deepfake technology has made the creation of child pornography faster and easier than it has ever been. Deepfakes can be used to produce new child pornography from already existing material or creating pornography from children who have not been subjected to sexual abuse. Deepfake child pornography can, however, have real and direct implications on children including defamation, grooming, extortion, and bullying.
Differences from generative AI pornography
While both deepfake pornography and generative AI pornography utilize synthetic media, they differ in approach and ethical implications. Generative AI pornography is created entirely through algorithms, producing hyper-realistic content unlinked to real individuals.
In contrast, deepfake pornography alters existing footage of real individuals, often without consent, by superimposing faces or modifying scenes. Hany Farid, a digital image analysis expert, has emphasized these distinctions.
Most deepfake pornography is made using the faces of people who did not consent to their image being used in such a sexual way. In 2023, Sensity, an identity verification company, has found that "96% of deepfakes are sexually explicit and feature women who didn't consent to the creation of the content".
Combatting deepfake pornography
Technical approach
Deepfake detection has become an increasingly important area of research in recent years as the spread of fake videos and images has become more prevalent. One promising approach to detecting deepfakes is through the use of Convolutional Neural Networks (CNNs), which have shown high accuracy in distinguishing between real and fake images. One CNN-based algorithm that has been developed specifically for deepfake detection is DeepRhythm, which has demonstrated an impressive accuracy score of 0.98 (i.e. successful at detecting deepfake images 98% of the time). This algorithm utilizes a pre-trained CNN to extract features from facial regions of interest and then applies a novel attention mechanism to identify discrepancies between the original and manipulated images. While the development of more sophisticated deepfake technology presents ongoing challenges to detection efforts, the high accuracy of algorithms like DeepRhythm offers a promising tool for identifying and mitigating the spread of harmful deepfakes.
Aside from detection models, there are also video authenticating tools available to the public. In 2019, Deepware launched the first publicly available detection tool which allowed users to easily scan and detect deepfake videos. Similarly, in 2020 Microsoft released a free and user-friendly video authenticator. Users upload a suspected video or input a link, and receive a confidence score to assess the level of manipulation in a deepfake.
Legal approach
Victims of deepfake pornography often have claims for revenge porn, tort claims, and harassment. The legal consequences for revenge porn vary from state to state and country to country. For instance, in Canada, the penalty for publishing non-consensual intimate images is up to 5 years in prison, whereas in Malta it is a fine of up to €5,000.
Australia
In April 2025, a 19 year old Australian, William Hamish Yeates, pleaded guilty to creating and distributing deepfake pornography, becoming the first person charged under Australia's new national laws targeting offenders manipulating sexual images. He admitted to multiple offences, including sharing altered images of a victim across social media, with some charges dropped following his plea.
Germany
According to Justice Minister Stefanie Hubig's spokesman, the law against "deepfake pornography" is ready and will be presented in a very short time, as of March 2026. The proposal aims to extend police authority to search suspect's devices and improve law enforcement, so offenders can be easily identified and prosecuted, addressing gaps where current law is insufficient.
South Korea
In South Korea, the creation, distribution, or possession of deepfake pornography is classified as a sex crime, with a mandatory prison sentence between three and seven years as part of the country's Special Act on Sexual Violence Crimes.
United Kingdom
In the United Kingdom, the Law Commission for England and Wales recommended reform to criminalise sharing of deepfake pornography in 2022. In 2023, the government announced amendments to the Online Safety Bill to that end. The Online Safety Act 2023 amends the Sexual Offences Act 2003 to criminalise sharing intimate images that shows or "appears to show" another (thus including deepfake images) without consent. In 2024, the Government announced that an offence criminalising the production of deepfake pornographic images would be included in the Criminal Justice Bill of 2024. The bill did not pass before Parliament was dissolved before the general election. The Data (Use and Access) Act 2025 introduced a new offence of creating, or requesting the creation of, "a purported intimate image of another person" without the explicit consent of the person depicted in the image.
United States
In the United States, the TAKE IT DOWN Act was signed into law in May 2025, which addressed non-consensual intimate images as well as deepfake pornography. In the United States, despite 38 states having laws regarding AI CSAM in 2025, several loopholes in enforcement remain, including the lack of definition of "student on student" AI generated deepfake child pornography.
The "Deepfake Accountability Act" was introduced to the United States Congress in 2019 but died in 2020. It aimed to make the production and distribution of digitally altered visual media that was not disclosed to be such, a criminal offense. The title specifies that making any sexual, non-consensual altered media with the intent of humiliating or otherwise harming the participants, may be fined, imprisoned for up to 5 years or both. A newer version of bill was introduced in 2021 which would have required any "advanced technological false personation records" to contain a watermark and an audiovisual disclosure to identify and explain any altered audio and visual elements. The bill also includes that failure to disclose this information with intent to harass or humiliate a person with an "advanced technological false personation record" containing sexual content "shall be fined under this title, imprisoned for not more than 5 years, or both." However this bill died in 2023. As of 2023, there was a lack of legislation that specifically addressed deepfake pornography. Instead, the harm caused by its creation and distribution was being addressed by the courts through existing criminal and civil laws.
On September 14, 2026, New York District Attorney Alvin Bragg announced that his office had seized twelve websites hosting nonconsensual AI-generated intimate imagery, the largest such seizure to date. Sexually explicit deepfakes had been criminalized in New York since 2023.
Controlling the distribution
While the legal landscape remains in development, victims of deepfake pornography have several tools available to contain and remove content, including securing removal through a court order, intellectual property tools (like DMCA takedowns in the United States), reporting for terms and conditions violations of the hosting platform, and removal by reporting the content to search engines.
Several major online platforms have taken steps to ban deepfake pornography. As of 2018, gfycat, reddit, Twitter, Discord, and Pornhub have all prohibited the uploading and sharing of deepfake pornographic content on their platforms. In September of that same year, Google also added "involuntary synthetic pornographic imagery" to its ban list, allowing individuals to request the removal of such content from search results.