Introduction

Artificial intelligence can algorithmically compose original, stylistically consistent musical works by applying sophisticated machine learning to analyse patterns in large music datasets. This post provides an in-depth technical explanation of how AI generates music, covering data preparation, feature extraction, modelling, generative techniques and audio synthesis.

Musical Data Preparation  

The first step is curating the training data that teaches the AI about musical style and structure. This involves collecting a large corpus of existing compositions in the target genre and formatting into symbolic notation data like MIDI or MusicXML.

MIDI (Musical Instrument Digital Interface) and MusicXML are two common file formats used for representing and transmitting musical score data between different music software and applications:

• MIDI is a technical standard protocol and file format that allows musical instruments, computers, and other equipment to connect and communicate with each other to play, edit, or record music. A MIDI file contains commands that specify musical notes, tempo, rhythm, velocity and other expressive details about the music. MIDI files do not contain the actual audio waveforms, only the instructions needed to recreate the music. MIDI is commonly used with electronic instruments, software synthesizers, sequencers, and more.

• MusicXML is an open file format designed specifically for representing sheet music notation using XML markup. It encodes the musical symbols and notation like notes, rhythms, dynamics, articulations, tempo, key, time signatures and more that are seen on sheet music. MusicXML files contain much richer musical information compared to MIDI files and are the preferred format for exchanging and distributing sheet music between various music notation software.

In comparing the two, MIDI represents musical performance information and instructions for playback, while MusicXML represents detailed printed sheet music notation. MIDI is more limited in capturing all the nuances of notation, while MusicXML can represent every aspect of sheet music with precision. Both formats are widely used in music production.

The training data should contain thousands of diverse scores covering the full stylistic breadth. Data is pre-processed via cleaning, structuring and augmentation. Experts manually annotate tracks with metadata to optimise machine learning.

Feature Extraction

Next, salient musical features that represent the essence of the training data are extracted. Low-level features describe pitch, melody, rhythm, tempo and harmony. Fourier analysis detects harmonic patterns. Markov models determine transition probabilities between musical elements.

Markov models are statistical models that are used to model randomly changing systems such as music. They work by assuming the future state of a system depends only on its current state, not on the sequence of events that preceded it. In a Markov model, the system being modelled is represented as states with probabilities associated with the transitions between each state. These transition probabilities are the core component of the model. By calculating probabilities of state transitions, they can predict the evolution of complex processes.