Artificial intelligence in music

Software applications

ChucK

Developed at Princeton University by Ge Wang and Perry Cook, ChucK is a text-based, cross-platform language. By extracting and classifying the theoretical techniques it finds in musical pieces, the software is able to synthesize entirely new pieces from the techniques it has learned. The technology is used by SLOrk (Stanford Laptop Orchestra) and PLOrk (Princeton Laptop Orchestra).

Jukedeck

Jukedeck was a website that let people use artificial intelligence to generate original, royalty-free music for use in videos. The team started building the music generation technology in 2010, formed a company around it in 2012, and launched the website publicly in 2015. The technology used was originally a rule-based algorithmic composition system, which was later replaced with artificial neural networks. The website was used to create over 1 million pieces of music, and brands that used it included Coca-Cola, Google, UKTV, and the Natural History Museum, London. In 2019, the company was acquired by ByteDance.

MorpheuS

MorpheuS is a research project by Dorien Herremans and Elaine Chew at Queen Mary University of London, funded by a Marie Skłodowská-Curie EU project. The system uses an optimization approach based on a variable neighborhood search algorithm to morph existing template pieces into novel pieces with a set level of tonal tension that changes dynamically throughout the piece. This optimization approach allows for the integration of a pattern detection technique in order to enforce long term structure and recurring themes in the generated music. Pieces composed by MorpheuS have been performed at concerts in both Stanford and London.

AIVA

Created in February 2016, in Luxembourg, AIVA is a program that produces soundtracks for any type of media. The algorithms behind AIVA are based on deep learning architectures AIVA has also been used to compose a Rock track called On the Edge, as well as a pop tune Love Sick in collaboration with singer Taryn Southern, for the creation of her 2018 album "I am AI".

Google Magenta

Google's Magenta team has published several AI music applications and technical papers since their launch in 2016. In 2017 they released the NSynth algorithm and dataset, and an open source hardware musical instrument, designed to facilitate musicians in using the algorithm. The instrument was used by notable artists such as Grimes and YACHT in their albums. In 2018, they released a piano improvisation app called Piano Genie. This was later followed by Magenta Studio, a suite of 5 MIDI plugins that allow music producers to elaborate on existing music in their DAW. In 2023, their machine learning team published a technical paper on GitHub that described MusicLM, a private text-to-music generator which they'd developed.

Riffusion

Riffusion is a neural network, designed by Seth Forsgren and Hayk Martiros, that generates music using images of sound rather than audio.

The resulting music has been described as "de otro mundo" (otherworldly), although unlikely to replace man-made music. The model was made available on December 15, 2022, with the code also freely available on GitHub.

The first version of Riffusion was created as a fine-tuning of Stable Diffusion, an existing open-source model for generating images from text prompts, on spectrograms, resulting in a model which used text prompts to generate image files which could then be put through an inverse Fourier transform and converted into audio files. While these files were only several seconds long, the model could also use latent space between outputs to interpolate different files together (using the img2img capabilities of SD). It was one of many models derived from Stable Diffusion.

In December 2022, Mubert similarly used Stable Diffusion to turn descriptive text into music loops. In January 2023, Google published a paper on their own text-to-music generator called MusicLM.

Forsgren and Martiros formed a startup, also called Riffusion, and raised $4 million in venture capital funding in October 2023.

Spike AI

Spike AI is an AI-based audio plug-in, developed by Spike Stent in collaboration with his son Joshua Stent and friend Henry Ramsey, that analyzes tracks and provides suggestions to increase clarity and other aspects during mixing. Communication is done by using a chatbot trained on Spike Stent's personal data. The plug-in integrates into digital audio workstation.

Musical applications

Artificial intelligence can potentially impact how producers create music by giving reiterations of a track that follow a prompt given by the creator. These prompts allow the AI to follow a certain style that the artist is trying to go for. AI has also been seen in musical analysis where it has been used for feature extraction, pattern recognition, and musical recommendations. New tools that are powered by artificial intelligence have been made to help aid in generating original music compositions, like AIVA (Artificial Intelligence Virtual Artist) and Udio. This is done by giving an AI model data of already-existing music and having it analyze the data using deep learning techniques to generate music in many different genres, such as classical music or electronic music. Musical and choral classrooms are already implementing AI driven programs to aid in musical learning, and development of creative skills in students where AI has shown to provide a statistically significant increase.

Ethical and legal considerations

Several musicians, such as Dua Lipa, Elton John, Nick Cave, Paul McCartney, and Sting, have criticized the use of AI in music and are encouraging the UK government to act on this matter. Another example of this protest is the silent 2025 album Is This What We Want?.

Some artists have encouraged the use of AI in music, such as Grimes.

While helpful in generating new music, many issues have come up since AI has begun making music. Some major concerns include how the economy will be impacted with AI taking over music production, who truly owns music generated by AI, and a lower demand for human-made musical compositions. Some critics argue that AI diminishes the value of human creativity, while proponents see it as an augmentative tool that expands artistic possibilities rather than replacing human musicians.

Additionally, concerns have been raised about AI's potential to homogenize music. AI-driven models often generate compositions based on existing trends, which some fear could limit musical diversity. Addressing this concern, researchers are working on AI systems that incorporate more nuanced creative elements, allowing for greater stylistic variation.

Another major concern about AI in music is copyright laws. In the United States, the current legal framework tends to apply traditional copyright laws to AI, despite its differences with the human creative process. However, music outputs solely generated by AI are not granted copyright protection. In the compendium of the U.S. Copyright Office Practices, the Copyright Office has stated that it would not grant copyrights to "works that lack human authorship" and "the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author." In February 2022, the Copyright Review Board rejected an application to copyright AI-generated artwork on the basis that it "lacked the required human authorship necessary to sustain a claim in copyright." The usage of copyrighted music in training AI has also been a topic of contention. One instance of this was seen when SACEM, a professional organization of songwriters, composers, and music publishers demanded that PozaLabs, an AI music generation startup refrain from utilizing any music affiliated with them for training models.

The situation in the European Union (EU) is similar to the US, because its legal framework also emphasizes the role of human involvement in a copyright-protected work. According to the European Union Intellectual Property Office and the recent jurisprudence of the Court of Justice of the European Union, the originality criterion requires the work to be the author's own intellectual creation, reflecting the personality of the author evidenced by the creative choices made during its production, requires distinct level of human involvement. The reCreating Europe project, funded by the European Union's Horizon 2020 research and innovation program, delves into the challenges posed by AI-generated contents including music, suggesting legal certainty and balanced protection that encourages innovation while respecting copyright norms. The recognition of AIVA marks a significant departure from traditional views on authorship and copyrights in the realm of music composition, allowing AI artists capable of releasing music and earning royalties. This acceptance marks AIVA as a pioneering instance where an AI has been formally acknowledged within the music production.

The recent advancements in AI made by groups, such as Stability AI, OpenAI, and Google, has incurred an enormous sum of copyright claims leveled against generative technology, including AI music. In one such example, Universal Music Group and Sony Music filed copyright infringement lawsuits against Suno, an AI music generation platform, alleging the company copied songs without permission to train its AI models. Should these lawsuits succeed, the machine learning models behind these technologies would have their datasets restricted to the public domain. Strides toward addressing ethical issues have been made as well, such as the collaboration between Sound Ethics (a company promoting ethical AI usage in the music industry) and UC Irvine, focusing on ethical frameworks and the responsible usage of AI. Internationally, systemic resistance against unauthorized generative models has escalated; in mid-2026, major entertainment coalitions in South Korea formed a unified front against AI copyright infringement. Market infrastructure analysts noted that this cross-border consolidation marks a critical shift toward protecting sovereign cultural assets, serving as a legislative blueprint for global intellectual property frameworks fighting against automated sonic commodification.

Musical deepfakes

A more nascent development of AI in music is the application of audio deepfakes to cast the lyrics or musical style of a pre-existing song to the voice or style of another artist. This has raised many concerns regarding the legality of technology, as well as the ethics of employing it, particularly in the context of artistic identity. Furthermore, it has also raised the question of to whom the authorship of these works is attributed. As AI cannot hold authorship of its own, current speculation suggests that there will be no clear answer until further rulings are made regarding machine learning technologies as a whole. Most recently, preventative measures have started to be developed by Google and Universal Music group who have taken in royalties and credited attribution in order to allow producers to replicate the voices and styles of artists.

"Heart on My Sleeve"

In 2023, an artist known as ghostwriter977 created a musical deepfake called "Heart on My Sleeve" that cloned the voices of Drake and The Weeknd by inputting an assortment of vocal-only tracks from the respective artists into a deep-learning algorithm, creating an artificial model of the voices of each artist, to which this model could be mapped onto original reference vocals with original lyrics. The track was submitted for Grammy consideration for the best rap song and song of the year. It went viral and gained traction on TikTok and received a positive response from the audience, leading to its official release on Apple Music, Spotify, and YouTube in April 2023. Many believed the track was fully composed by an AI software, but the producer claimed the songwriting, production, and original vocals (pre-conversion) were still done by him. It would later be rescinded from any Grammy considerations due to it not following the guidelines necessary to be considered for a Grammy award. The track would end up being removed from all music platforms by Universal Music Group. The song was a watershed moment for AI voice cloning, and models have since been created for hundreds, if not thousands, of popular singers and rappers.

"Where That Came From"

In 2013, country music singer Randy Travis suffered a stroke which left him unable to sing. In the meantime, vocalist James Dupré toured on his behalf, singing his songs for him. Travis and longtime producer Kyle Lehning released a new song in May 2024 titled "Where That Came From", Travis's first new song since his stroke. The recording uses AI technology to re-create Travis's singing voice, having been composited from over 40 existing vocal recordings alongside those of Dupré.

Kanye West: Vultures 2, Bully, Cuck, and Donda 2

Since 2024, rapper Kanye West has been using artificial intelligence deepfakes of his own voice. His usage of deepfakes started during the production of his album Vultures 2, where the songs "Field Trip" and "Sky City" drew suspicion of artificial intelligence usage; further updates to the songs "Forever" and "530" would also be accused of using AI. Artist Ty Dolla Sign, who released Vultures 2 with West, confirmed the allegations in 2025 during an interview. West subsequently confirmed that his subsequent solo album, Bully, also used artificial intelligence. Though West confirmed in 2026 that he would remove the AI, the physical versions of the album still contained it.

His unreleased album Cuck and the 2025 re-release of Donda 2 would also draw accusations of AI usage.

Playboi Carti: "Timeless" and Music

Playboi Carti has also been accused of using artificial intelligence deepfakes. The allegations followed the release of the song "Timeless", a collaboration with The Weeknd, where his verse was accused of using artificial intelligence. The allegations would further intensify after the release of his album Music, where "Rather Lie" and "Fine Shit" were also accused of using artificial intelligence. Playboi Carti would deny the accusations.

Original Pilipino Music (OPM)

Numerous OPM music have had notable AI covers. An AI cover of "Pahina" by Cup of Joe hit the Spotify Philippines' Viral Songs chart, with a negative reaction from one of the band members, and has since been removed from Spotify. Lofi Town, an AI artist covering Filipino songs, also hit Spotify Philippines' Top Artists list, with negative reactions from several OPM artists, including Maki, Adie, mrld, TJ Monterde, Janine Teñoso, and Angela Ken. An AI cover of "Kay Tagal Kitang Hinintay" was also criticized by Karylle.

Technical approaches

Artificial intelligence music encompasses a number of technical approaches used for music composition, analysis, classification, and suggestion. Techniques used are drawn from deep learning, machine learning, natural language processing, and signal processing. Current systems are able to compose entire musical compositions, parse affective content, accompany human players in real-time, and acquire patterns of user and context-dependent preferences.

Symbolic music composition

Symbolic music generation is the generation of music in discrete symbolic forms such as MIDI, where note and timing are precisely defined. Early systems employed rule-based systems and Markov models, but modern systems employ deep learning to a large extent. Recurrent Neural Networks (RNNs), and more precisely Long Short-Term Memory (LSTM) networks, have been employed in modeling temporal dependencies of musical sequences. They may be used to generate melodies, harmonies, and counterpoints in various musical genres.

Transformer models such as Music Transformer and MuseNet became more popular for symbolic generation due to their ability to model long-range dependencies and scalability. These models were employed to generate multi-instrument polyphonic music and stylistic imitations.

Audio-based music generation

This method generates music as raw audio waveforms instead of symbolic notation. DeepMind's WaveNet is an early example that uses autoregressive sampling to generate high-fidelity audio. Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) are being used more and more in new audio texture synthesis and timbre combination of different instruments.

NSynth (Neural Synthesizer), a Google Magenta project, uses a WaveNet-like autoencoder to learn latent audio representations and thereby generate completely novel instrumental sounds.

Music information retrieval (MIR)

Music Information Retrieval (MIR) is the extraction of musically relevant information from audio recordings to be utilized in applications such as genre classification, instrument recognition, mood recognition, beat detection, and similarity estimation. CNNs on spectrogram features have been very accurate on these tasks. SVMs and k-Nearest Neighbors (k-NN) are also used for classification on features such as Mel-frequency cepstral coefficients (MFCCs).

Hybrid and interactive systems

Hybrid systems combine symbolic and sound-based methods to draw on their respective strengths. They can compose high-level symbolic compositions and synthesize them as natural sound. Interactive systems in real-time allow for AI to instantaneously respond to human input to support live performance. Reinforcement learning and rule-based agents tend to be utilized to allow for human–AI co-creation in improvisation contexts.

Affective computing and emotion-aware music systems

Affective computing techniques enable AI systems to classify or create music based on some affective content. The models use musical features such as tempo, mode, and timbre to classify or influence listener emotions. Deep learning models have been trained for classifying music based on affective content and even creating music intended to have affective impacts.

AI-based music recommendation systems

Music recommenders employ AI to suggest tracks to users based on what they have heard, their tastes, and information available in context. Collaborative filtering, content-based filtering, and hybrid filtering are most widely applied, deep learning being utilized for fine-tuning. Graph-based and matrix factorization methods are used within commercial systems like Spotify and YouTube Music to represent complex user-item relationships.

AI for automatic mixing and mastering

AI is also used in audio engineering automation such as mixing and mastering. Such systems level, equalize, pan, and compress to give well-balanced sound outputs. Software such as LANDR and iZotope Ozone utilize machine learning in emulating professional audio engineers' decisions.

Lyrics generation and songwriting aid

Natural language generation also applies to songwriting assistance and lyrics generation. Transformer language models like GPT-3 have also been proven to be able to generate stylistic and coherent lyrics from input prompts, themes, or feeling. There even exist AI programs that assist with rhyme scheme, syllable count, and poem form.

Multimodal and cross-modal systems

Recent developments include multimodal AI systems that integrate music with other media, e.g., dance, video, and text. These can generate background scores in synchronization with video sequences or generate dance choreography from audio input. Cross-modal retrieval systems allow one to search for music using images, text, or gestures.