Generative AI · Lablypedia

The prevalence of generative AI tools has increased significantly since the AI boom of the early 2020s. This boom was made possible by improvements in deep neural networks, particularly large language models (LLMs), which are based on the transformer architecture. Generative AI applications include chatbots such as ChatGPT, Claude, Microsoft Copilot, DeepSeek, Doubao, Google Gemini, Grok, Kimi and Qwen; text-to-image models such as DALL-E, Firefly, Stable Diffusion, and Midjourney; and text-to-video models such as Veo, LTX and Sora.

Companies in a variety of sectors have used or proposed the use of generative AI, including those in software development, healthcare, finance, entertainment, sales and marketing, screenwriting, and product design.

Generative AI has been used for cybercrime, and to deceive and manipulate people through fake news and deepfakes. Generative AI models have been trained on copyrighted works without the rightholders' permission. Many generative AI systems are reliant on large-scale data centers, whose environmental impacts include electronic waste, consumption of fresh water for cooling, and high energy consumption that is estimated to be growing steadily.

History

Early history

The origins of algorithmically generated media can be traced to the development of the Markov chain, which has been used to model natural language since the early 20th century. Russian mathematician Andrey Markov introduced the concept in 1906, including an analysis of vowel and consonant patterns in Eugene Onegin. Once trained on a text corpus, a Markov chain can generate probabilistic text.

By the early 1970s, artists began using computers to extend generative techniques beyond Markov models. Harold Cohen developed and exhibited works produced by AARON, a pioneering computer program designed to autonomously create paintings. The terms generative AI planning or generative planning were used in the 1980s and 1990s to refer to AI planning systems, especially computer-aided process planning, used to generate sequences of actions to reach a specified goal. Generative AI planning systems used symbolic AI methods such as state space search and constraint satisfaction and were a "relatively mature" technology by the early 1990s. They were used to generate crisis action plans for military use, process plans for manufacturing and decision plans such as in prototype autonomous spacecraft.

Generative neural networks (since the late 2000s)

Machine learning uses both discriminative models and generative models to predict or generate data. Beginning in the late 2000s and early 2010s, advances in deep learning led to major improvements in image classification, speech recognition, and natural language processing. Neural networks in this period were typically trained as discriminative models due to the relative difficulty of training generative models.

In 2014, the introduction of models such as the variational autoencoder (VAE) and generative adversarial network (GAN) enabled effective deep generative modeling of complex data such as images. In 2017, the paper "Attention Is All You Need" proposed that transformer architecture could enable further advances in generative modeling compared to earlier long short-term memory (LSTM) networks. This led to the development of generative pre-trained transformer (GPT) models, beginning with GPT-1 in 2018.

Generative AI adoption

In March 2020, the release of 15.ai, a free web application created by an anonymous MIT researcher that could generate convincing character voices using minimal training data, was one of the earliest publicly available uses for generative AI. The platform is credited as the first mainstream service for audio deepfakes.

In 2021, DALL-E, a closed-source transformer-based generative model developed by OpenAI, drew widespread attention to text-to-image generation.

Other projects, including open-source approaches such as VQGAN+CLIP and DALL·E Mini (later renamed Craiyon), made similar systems more accessible to the public.

Dream by Wombo was released at the end of 2021, followed by the releases of Midjourney and Stable Diffusion in 2022.

In November 2022, ChatGPT was released to the public. By 2023, it popularized generative AI for general-purpose text-based tasks.

In a 2024 survey by marketing research firm Ipsos, Asia–Pacific countries were significantly more optimistic than Western societies about generative AI and show higher adoption rates. Despite expressing concerns about privacy and the pace of change, 68% of Asia-Pacific respondents believed that AI was having a positive impact on the world, compared to 57% globally. According to a survey by SAS and Coleman Parkes Research, as of 2023, 83% of Chinese respondents were using the technology, exceeding both the global average of 54% and the U.S. rate of 65%. A UN report indicated that Chinese entities filed over 38,000 generative AI patents from 2014 to 2023, more than any other country. A 2024 survey by the Just So Soul social media app reported that 18% of respondents born after 2000 used generative AI "almost every day", and that over 60% of respondents like or love AI-generated content (AIGC), while less than 3% dislike or hate it.

By mid-2025, companies were increasingly abandoning generative AI pilot projects as they had difficulties with integration, data quality and unmet returns, leading analysts at Gartner and The Economist to characterize the period as entering the Gartner hype cycle's "trough of disillusionment" phase.

In an October 2025 survey by the Federal Reserve, one in four U.S. workers reported using generative AI at work during the previous month.

5 more chapters are free with a Lably account.

Applications

Generative artificial intelligence has been applied across multiple industries for content creation and automation. In healthcare, generative models are used for drug discovery and the generation of synthetic medical data to train diagnostic systems. In finance, they are used for report drafting, data generation, and customer service automation. Media and entertainment industries use generative systems for tasks such as music composition, script development, and image or video generation. Researchers and policymakers have raised concerns regarding accuracy, misuse, and impacts on academic and professional work.

Text and software code

Large language models (LLMs) are trained on tokenized text from large corpora and are capable of natural language processing, machine translation, and natural language generation.

LLMs can be used as foundation models for a variety of downstream tasks. They can also be trained on source code to generate programs from prompts.

Many applications combine large language models with external knowledge sources using retrieval-augmented generation (RAG), a technique in which relevant documents are retrieved at inference time and incorporated into the model's response.

Generative AI has been integrated into content management and website publishing platforms, where large language models are used to generate text, assist with page composition, and automate website creation from natural language prompts. WordPress.com is one example of this application, incorporating generative AI into its website authoring workflow.

Audio

In 2016, DeepMind's WaveNet demonstrated that deep neural networks can generate raw audio waveforms. This enabled more realistic speech synthesis compared to earlier approaches. Subsequent systems such as Tacotron 2 demonstrated end-to-end neural text-to-speech generation.

Images

Generative AI can be used to create visual art. Such systems are trained on image–text pairs. Examples include Stable Diffusion, DALL-E, and Midjourney.

Video

Generative AI can be used to produce photorealistic videos. Systems such as Runway have demonstrated text-to-video generation capabilities.

Robotics

Generative models can be used for motion planning and robot control by learning from prior data.

3D modeling

Generative models can assist in automating 3D modeling tasks, including generating 3D assets from text or images.

AI-assisted mathematical discovery

Generative AI systems have been used in mathematics and computer science to generate candidate computer programs, proofs, constructions, or algorithms.

In 2023, Google DeepMind introduced FunSearch, a method for creating computer programs that solve mathematical and algorithmic problems. FunSearch was used to discover new mathematical constructions in the cap set problem and the bin packing problem.

In 2023, Google DeepMind introduced AlphaDev, which was used to discover small sorting algorithms that outperformed previously known human benchmarks and have been integrated into the LLVM standard C++ sorting library. In 2025, Google DeepMind introduced AlphaEvolve, an AI system for general-purpose algorithm discovery and optimization. AlphaEvolve uses LLMs to propose code changes, automated evaluators to assess each candidate, and an evolutionary process to iteratively improve algorithms.

In 2026, in response to the increasing use of generative AI in mathematical discovery, a group of mathematicians issued the Leiden Declaration on Artificial Intelligence and Mathematics, which recommends disclosing the use of AI in research papers, ensuring that AI-assisted papers are peer-reviewed, and providing legal resources and public funding so that academia and for-profit companies can compete on equal terms.

Materials science

In 2023, Google DeepMind introduced GNoME, a method to propose candidate inorganic crystal structures for computational screening and experimental synthesis in material science. Other material science methods include MatterGen, CDVAE, and CrystalFlow.

Generative engine optimization

Generative engine optimization (GEO) is the practice of structuring digital content and managing online presence to improve visibility in responses generated by generative AI systems. The practice influences the way large language models (LLMs) retrieve, summarize, and present information in response to user queries. Related terms include answer engine optimization (AEO) and artificial intelligence optimization (AIO).

Software and hardware

Generative AI models are used by chatbots such as ChatGPT, programming tools such as GitHub Copilot, text-to-image programs such as Midjourney, and text-to-video programs such as Runway Gen-2. Generative AI features have been added to existing commercially available services such as Microsoft Office (Microsoft Copilot), Google Photos, and the Adobe Suite (Adobe Firefly). Many generative AI models are also available as open-source software, including Stable Diffusion and the LLaMA language model.

Smaller generative AI models with up to a few billion parameters can run on smartphones, embedded devices, and personal computers. For example, LLaMA-7B (a version with 7 billion parameters) can run on a Raspberry Pi 4 and one version of Stable Diffusion can run on an iPhone 11.

Larger models with tens of billions of parameters can run on laptop or desktop computers. To achieve an acceptable speed, models of this size may require accelerators such as the GPU chips produced by NVIDIA and AMD or the Neural Engine included in Apple silicon products. For example, the 65 billion parameter version of LLaMA can be configured to run on a desktop PC.

The advantages of running generative AI locally include protection of privacy and intellectual property, and avoidance of rate limiting and censorship. The subreddit r/LocalLLaMA in particular focuses on using consumer-grade gaming graphics cards through such techniques as compression.

Language models with hundreds of billions of parameters, such as GPT-4 or PaLM, typically run on datacenter computers equipped with arrays of GPUs (such as NVIDIA's H100) or AI accelerator chips (such as Google's TPU). These very large models are typically accessed as cloud services over the Internet.

In 2022, the United States New Export Controls on Advanced Computing and Semiconductors to China imposed restrictions on exports to China of GPU and AI accelerator chips used for generative AI. Chips such as the NVIDIA A800 and the Biren Technology BR104 were developed to meet the requirements of the sanctions.

There is free software on the market which attempts to recognizing text generated by generative artificial intelligence (such as GPTZero), as well as images, audio or video coming from it. Potential mitigation strategies for detecting generative AI content include digital watermarking, content authentication, information retrieval, and machine learning classifier models. Despite claims of accuracy, both free and paid AI text detectors have frequently produced false positives, mistakenly accusing students of submitting AI-generated work.

Generative models and training techniques

Generative adversarial networks

Generative adversarial networks (GANs) are a generative modeling technique which consist of two neural networks—the generator and the discriminator—trained simultaneously in a competitive setting. The generator creates synthetic data by transforming random noise into samples that resemble the training dataset. The discriminator is trained to distinguish the authentic data from synthetic data produced by the generator. The two models engage in a minimax game: the generator aims to create increasingly realistic data to "fool" the discriminator, while the discriminator improves its ability to distinguish real from fake data. This continuous training setup enables the generator to produce high-quality and realistic outputs.

Variational autoencoders

Variational autoencoders (VAEs) are deep learning models that probabilistically encode data. They are typically used for tasks such as noise reduction from images, data compression, identifying unusual patterns, and facial recognition. Unlike standard autoencoders, which compress input data into a fixed latent representation, VAEs model the latent space as a probability distribution, allowing for smooth sampling and interpolation between data points. The encoder ("recognition model") maps input data to a latent space, producing means and variances that define a probability distribution. The decoder ("generative model") samples from this latent distribution and attempts to reconstruct the original input.

Transformers

The self-attention mechanism enables the model to determine the relative importance of each token in a sequence when predicting the next token, thereby improving contextual understanding. Unlike recurrent neural networks, transformers process tokens in parallel, which improves training efficiency and scalability. Transformers became the foundation for the generative pre-trained transformer (GPT) series developed by OpenAI, replacing traditional recurrent and convolutional models.

Law and regulation

In the United States, a group of companies including OpenAI, Alphabet, and Meta signed a voluntary agreement with the Biden administration in July 2023 to watermark AI-generated content. In October 2023, Executive Order 14110 applied the Defense Production Act to require all US companies to report information to the federal government when training certain high-impact AI models.

In the European Union (EU), the Artificial Intelligence Act includes requirements to disclose copyrighted material used to train generative AI systems, and to label any AI-generated output as such.

In China, the Interim Measures for the Management of Generative AI Services introduced by the Cyberspace Administration of China regulates any public-facing generative AI. It includes requirements to watermark generated images or videos, regulations on training data and label quality, restrictions on personal data collection, and a guideline that generative AI services must "adhere to socialist core values".

In 2021, UNESCO adopted the Recommendation on the Ethics of Artificial Intelligence, the first global standard-setting instrument on AI. It provides guidance to member states on issues including human rights, fairness, transparency, data governance, and environmental sustainability.

The Group of Seven (G7) launched the Hiroshima AI Process in 2023, which produced international guiding principles and a voluntary code of conduct for developers of advanced AI systems. In 2024, the Council of Europe adopted the Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, the first legally binding international treaty governing artificial intelligence.

Copyright

Training with copyrighted content

Generative AI systems such as ChatGPT and Midjourney are trained on large, publicly available datasets that include copyrighted works. AI developers have argued that such training is protected under fair use, while copyright holders have argued that it infringes their rights.

Proponents of fair use training have argued that it is a transformative use and does not involve making copies of copyrighted works available to the public. Critics have argued that image generators such as Midjourney can create nearly-identical copies of some copyrighted images, and that generative AI programs compete with the content they are trained on.

As of 2024, several lawsuits related to the use of copyrighted material in training are ongoing. Getty Images has sued Stability AI over the use of its images to train Stable Diffusion. Both the Authors Guild and The New York Times have sued Microsoft and OpenAI over the use of their works to train ChatGPT.

Copyright of AI-generated content

A separate question is whether AI-generated works can qualify for copyright protection. The United States Copyright Office has ruled that works created by artificial intelligence without any human input cannot be copyrighted, because they lack human authorship. Some legal professionals have suggested that Naruto v. Slater (2018), in which the U.S. 9th Circuit Court of Appeals held that non-humans cannot be copyright holders of artistic works, could be a potential precedent in copyright litigation over works created by generative AI. However, the office has also begun taking public input to determine if these rules need to be refined for generative AI.

In January 2025, the United States Copyright Office (USCO) released extensive guidance regarding the use of AI tools in the creative process, and established that "...generative AI systems also offer tools that similarly allow users to exert control. [These] can enable the user to control the selection and placement of individual creative elements. Whether such modifications rise to the minimum standard of originality required under Feist will depend on a case-by-case determination. In those cases where they do, the output should be copyrightable" Subsequently, the USCO registered the first visual artwork to be composed of entirely AI-generated materials, titled "A Single Piece of American Cheese".

Ethical concerns

The development of generative AI has raised concerns from governments, businesses, and individuals, resulting in protests, legal actions, calls to pause AI experiments, and actions by multiple governments. In a July 2023 briefing of the United Nations Security Council, Secretary-General António Guterres stated "Generative AI has enormous potential for good and evil at scale", that AI may "turbocharge global development" and contribute between $10 and $15 trillion to the global economy by 2030, but that its malicious use "could cause horrific levels of death and destruction, widespread trauma, and deep psychological damage on an unimaginable scale". In addition, generative AI has a significant carbon footprint.

Societal impacts

Effects on mental health

Based on a nationally representative sample of 20,847 US adults, a 2026 survey study in JAMA Network Open found that daily generative AI use was significantly associated with greater depressive symptoms, with the odds of at least moderate depression roughly 30% higher among daily users than among non-users. The association was strongest for personal use and among those aged 25 to 64; the cross-sectional design did not allow for causal conclusions. Separately, a 2025 nationally representative survey of US adolescents and young adults (aged 12 to 21) found that roughly 13% had used generative AI for emotional support, with more than 90% of them reporting the advice as helpful.

Controlled trials have shown some therapeutic potential. The first randomized controlled trial of a fully generative AI therapy chatbot, published in NEJM AI in 2025, found significant reductions in symptoms of depression, anxiety, and eating disorders over four weeks compared to a waitlist control, with participants rating the therapeutic alliance as comparable to that of human therapists. These findings are consistent with a 2025 meta-analysis of 31 RCTs (n=29,637) that found small-to-moderate effects of AI chatbots in reducing depressive, anxiety, and stress symptoms among young adults. Commercially available therapy chatbots nevertheless show significant shortcomings: a 2025 Stanford University study found that they exhibited stigmatizing attitudes toward conditions such as schizophrenia and alcohol dependence, and responded inappropriately to suicidal ideation and delusional thinking.

Researchers have also raised concerns about AI companion chatbots and emotional dependency. While some users of companion apps report reduced loneliness, scientists worry about long-term dependency when chatbots provide constant validation without challenging users' assumptions. In 2026, Stanford researchers found that users of the Character.AI platform with limited offline social networks reported feeling more lonely after seeking emotional support from chatbots.

Overreliance on generative AI may also affect cognitive skills. In a study presented at the 2026 Conference on Human Factors in Computing Systems, participants who used AI regularly over four weeks showed a decreased ability to discern misinformation on their own.

Academic honesty

Generative AI can be used to generate and modify academic prose, paraphrase sources, and translate languages. The use of generative AI in a classroom setting has challenged traditional definitions of academic plagiarism, leading to a "cat-and-mouse" dynamic between students using AI and institutions attempting to detect it. In the immediate wake of ChatGPT's release, many school districts and universities issued temporary bans on the technology, though many institutions have since moved toward policies of managed integration. However, the implementation of these policies often lacks clarity. Research suggests that the burden of interpreting "acceptable use" frequently falls on individual students and teachers, creating an environment where academic honesty becomes difficult to define and enforce.

A commonly proposed use for teachers is grading and giving feedback. Companies like Pearson and ETS use AI to score grammar, mechanics, usage, and style, but not for main ideas or overall structure. The National Council of Teachers of English stated that machine scoring makes students feel their writing is not worth reading. AI scoring has also given unfair results for students from different ethnic backgrounds.

Fears over job losses

From the early days of the development of AI, there have been arguments put forward by ELIZA creator Joseph Weizenbaum and others about whether tasks that can be done by computers actually should be done by them, given the difference between computers and humans, and between quantitative calculations and qualitative, value-based judgements. In April 2023, it was reported that image generation AI has resulted in 70% of the jobs for video game illustrators in China being lost. In July 2023, developments in generative AI contributed to the 2023 Hollywood labor disputes. Fran Drescher, president of the Screen Actors Guild, declared that "artificial intelligence poses an existential threat to creative professions" during the 2023 SAG-AFTRA strike. Voice generation AI has been seen as a potential challenge to the voice acting sector.

However, a 2025 study concluded that the US labor market had so far not experienced a discernible disruption from generative AI. Another study reported that Danish workers who used chatbots saved 2.8% of their time on average, and found no significant change in earnings or hours worked.

Effects on journalism

In January 2023, Futurism broke the story that CNET had been using an undisclosed internal AI tool to write at least 77 of its stories; after the news broke, CNET posted corrections to 41 of the stories. In April 2023, Die Aktuelle published an AI-generated fake interview of Michael Schumacher. In May 2024, Futurism noted that a content management system video by AdVon Commerce, which had used generative AI to produce articles for many of the aforementioned outlets, appeared to show that they "had produced tens of thousands of articles for more than 150 publishers". In 2025, a report from the American Sunlight Project stated that Pravda network was publishing as many as 10,000 articles a day, and concluded that much of this content aimed to push Russian narratives into large language models through their training data.

In June 2024, Reuters Institute published its Digital News Report for 2024. In a survey of people in America and Europe, Reuters Institute reports that 52% and 47% respectively are uncomfortable with news produced by "mostly AI with some human oversight", and 23% and 15% respectively report being comfortable. 42% of Americans and 33% of Europeans reported that they were comfortable with news produced by "mainly human with some help from AI". The results of global surveys reported that people were more uncomfortable with news topics including politics (46%), crime (43%), and local news (37%) produced by AI than other news topics. A 2025 Pew Research Survey found roughly half of all U.S. adults say that AI will have a very (24%) or somewhat (26%) negative impact on the news people get in the U.S. over the next 20 years.

Bias

A language model may associate certain professions with specific genders if such patterns are prevalent in the data. Similarly, some image generation systems have been observed to generate images of men as CEOs more often than women.

AI software, when using voice recognition software in particular, struggles to recognize and understand speech impediments. For example, people with a stutter struggle to activate voice-activated assistants such as Gemini and Siri due to how the software was trained.

Companies that use AI systems to hire for new positions also filter out people with accents and speech due to voice recognition software incorrectly transcribing how candidates speak during the interview process. Because of this, people with disabilities and uncommon accents don't often make it to a human interviewer when these generative AI systems are used. This is due to many AI models being trained and produced in the United States, and therefore, off of American accents.

Misinformation and disinformation

Deepfakes

Deepfakes (a portmanteau of "deep learning" and "fake") are AI-generated media that take a person in an existing image or video and replace them with someone else's likeness using artificial neural networks. Deepfakes have garnered widespread attention and concerns for their uses in deepfake celebrity pornographic videos, revenge porn, fake news, hoaxes, health disinformation, financial fraud, and covert foreign election interference.

In July 2023, the fact-checking company Logically found that the popular generative AI models Midjourney, DALL-E 2 and Stable Diffusion would produce plausible disinformation images when prompted to do so, such as images of electoral fraud in the United States and Muslim women supporting India's Bharatiya Janata Party.

Audio deepfakes

Instances of users abusing software to generate controversial statements in the vocal style of celebrities, public officials, and other famous individuals have raised ethical concerns over voice generation AI. In response, companies such as ElevenLabs have stated that they would work on mitigating potential abuse through safeguards and identity verification.

Concerns and fandoms have spawned from AI-generated music. The same software used to clone voices has been used on famous musicians' voices to create songs that mimic their voices, gaining both tremendous popularity and criticism. Similar techniques have also been used to create improved quality or full-length versions of songs that have been leaked or have yet to be released.

Information laundering

Generative AI has been noted for its use by state-sponsored propaganda campaigns in information laundering. According to a 2025 report by Graphika, generative AI is used to launder articles from Chinese state media such as China Global Television Network through various social media sites in an attempt to disguise the articles' origin.

Accuracy and content quality

AI's propensity to produce inaccurate content has led to an estimated global enterprise loss at over $67 million, though other research says the true impact is significant but not yet well-quantified. According to McKinsey’s State of AI 2024 survey, about 44% of organizations have experienced at least one negative consequence from GenAI, with inaccuracy and hallucination among the top cited risks.

Numerous studies have found AI consistently creates inaccurate summaries that sound confidently correct.

The New York Times defines slop as analogous to spam: "shoddy or unwanted A.I. content in social media, art, books, and ... in search results." Journalists have expressed concerns about the scale of low-quality generated content with respect to social media content moderation, the monetary incentives from social media companies to spread such content, false political messaging, spamming of scientific research paper submissions, increased time and effort to find higher quality or desired content on the Internet, the indexing of generated content by search engines, and on journalism itself. Studies have found that AI can create inaccurate claims, citations or summaries that sound confidently correct, a phenomenon called hallucination.

A paper published by researchers at Amazon Web Services AI Labs found that over 57% of sentences from a sample of over 6 billion sentences from Common Crawl, a snapshot of web pages, were machine translated. Many of these automated translations were seen as lower quality, especially for sentences that were translated into at least three languages. Many lower-resource languages (ex. Wolof, Xhosa) were translated across more languages than higher-resource languages (ex. English, French).

In September 2024, Robyn Speer, the author of wordfreq, an open source database that calculated word frequencies based on text from the Internet, announced that she had stopped updating the data for several reasons: high costs for obtaining data from Reddit and Twitter, excessive focus on generative AI compared to other methods in the natural language processing community, and that "generative AI has polluted the data".

The adoption of generative AI tools led to an explosion of AI-generated content across multiple domains. A study from University College London estimated that in 2023, more than 60,000 scholarly articles—over 1% of all publications—were likely written with LLM assistance. According to Stanford University's Institute for Human-Centered AI, approximately 17.5% of newly published computer science papers and 16.9% of peer review text now incorporate content generated by LLMs.

If AI-generated content is included in new data crawls from the Internet for additional training of AI models, defects in the resulting models may occur. Training an AI model exclusively on the output of another AI model produces a lower-quality model. Repeating this process, where each new model is trained on the previous model's output, leads to progressive degradation and eventually results in a "model collapse" after multiple iterations.

On the other side, synthetic data can be deployed to train machine learning models while preserving user privacy. The approach is not limited to text generation; image generation has been employed to train computer vision models.

Malicious use

Illegal imagery

Many websites that allow explicit AI generated images or videos have been created, and this has been used to create illegal content, such as rape, child sexual abuse material, necrophilia, and zoophilia.

Cybercrime

Generative AI's ability to create realistic fake content has been exploited in numerous types of cybercrime, including phishing scams. Deepfake video and audio have been used to create disinformation and fraud. In 2020, former Google click fraud czar Shuman Ghosemajumder argued that once deepfake videos become perfectly realistic, they would stop appearing remarkable to viewers, potentially leading to uncritical acceptance of false information. Additionally, large language models and other forms of text-generation AI have been used to create fake reviews of e-commerce websites to boost ratings. Cybercriminals have created large language models focused on fraud, including WormGPT and FraudGPT.

A 2023 study showed that generative AI can be vulnerable to jailbreaks, reverse psychology and prompt injection attacks, enabling attackers to obtain help with harmful requests, such as for crafting social engineering and phishing attacks. Additionally, other researchers have demonstrated that open-source models can be fine-tuned to remove their safety restrictions at low cost.

RAG poisoning

In 2025, Israel signed a $6 million contract with the US-based firm Clock Tower X that aimed to influence ChatGPT, Gemini and Grok by spreading pro-Israel information onto social media and websites. This was in an attempt to take advantage of the retrieval-augmented generation (RAG) technique which is used by LLMs to provide more up-to-date information.

Privacy and data governance

Extraterritorial data access

The CLOUD Act allows United States authorities to request data from covered service providers, including some AI service providers, regardless of where the data is physically stored. Courts can require parent companies to provide data held by their subsidiaries, and such orders may be accompanied by nondisclosure requirements preventing the provider from notifying affected users. This framework has been described in legal commentary as creating legal tension with Article 48 of the General Data Protection Regulation (GDPR), which restricts the transfer of personal data in response to foreign court or administrative orders unless based on an international agreement. As a result, service providers operating in both jurisdictions may face competing legal obligations under U.S. and EU law.

Environmental and industry impacts

Energy and environment

AI has a significant carbon footprint due to growing energy consumption from both training and usage. Scientists and journalists have expressed concerns about the environmental impact that the development and deployment of generative models are having: high CO₂ emissions, large amounts of freshwater used for data centers, high amounts of electricity usage, electronic waste, and pollution due to backup diesel generator exhaust. There is also concern that these impacts may increase as these models are incorporated into widely used search engines such as Google Search and Bing, as chatbots and other applications become more popular, and as models need to be retrained.

The carbon footprint of generative AI globally is estimated to be growing steadily, with potential annual emissions ranging from 18.21 to 245.94 million tons of CO₂ by 2035, with the highest estimates for 2035 nearing the impact of the United States beef industry on emissions (currently estimated to emit 257.5 million tons annually as of 2024).

Proposed mitigation strategies include factoring potential environmental costs prior to model development or data collection, increasing efficiency of data centers to reduce electricity/energy usage, building more efficient machine learning models, minimizing the number of times that models need to be retrained, developing a government-directed framework for auditing the environmental impact of these models, regulating for transparency of these models, regulating their energy and water usage, encouraging researchers to publish data on their models' carbon footprint, and increasing the number of subject matter experts who understand both machine learning and climate science.

Reliance on industry giants

Training frontier AI models requires an enormous amount of computing power. Usually only Big Tech companies have the financial resources to make such investments. Smaller start-ups such as Cohere and OpenAI end up buying access to data centers from Google and Microsoft respectively.

Detection and awareness

Tools such as GPTZero can detect content generated by AI. However, they can also make false accusations (false positives). Digital watermarking is a technique that improves detection accuracy. It works by altering the generated content at the source, in subtle ways which can be detected by corresponding software.

In 2023, OpenAI developed a watermarking tool for ChatGPT. However, it was not released, because they worried that users would switch to competitors. They also argued that it would be easy to circumvent, for example by asking another AI to rephrase.

In March 2025, China issued requirements for online service providers to label AI content.

In May 2025, Google deployed its watermarking tool, SynthID. It marks output from Gemini (text), Imagen (images), and Veo (video). To detect output from these products, one uses Google's "SynthID detector" portal.

In 2025, users mistakenly accused gaming companies of using generative AI for the video games Little Droid and Chessplus.