What just happened? Artificial intelligence has completed Beethoven's Tenth Symphony—or at least, what that piece of music could have sounded like had he finished it. A world premiere of the music will take place on October 9, 2021.
In 1817, the Royal Philharmonic Society in London commissioned Ludwig Von Beethoven to compose his Ninth and Tenth symphonies. He completed the Ninth Symphony, which concludes the fourth movement with the all too familiar "Ode to Joy" (below). Due to declining health and his subsequent death in 1827, he never completed his Tenth Symphony. All that remains of that work is some scrawled musical sketches.
The Karajan Institute partnered with a group of scientists from AI startup Playform led by Ahmed Elgammal. The team spent two years training an AI model using Beethoven's complete works, the Tenth Symphony sketches he left behind, and what is known about his methods of composing music.
Elgammal tapped Walter Werzowa, the mind behind Intel's bong jingle, to merge what Beethoven left behind with the AI-constructed music. Computational music expert Mark Gotham led efforts to transcribe the centuries-old sketches and process Beethoven's entire work to train machine learning algorithms. Harvard musicologist Robert Levin, who had previously completed several musical compositions from Mozart and Bach, also helped on the project.
"In a large room with a piano, a blackboard and a stack of Beethoven's sketchbooks spanning most of his known works, we talked about how fragments could be turned into a complete piece of music and how AI could help solve this puzzle, while still remaining faithful to Beethoven's process and vision," Elgammal said.
The human side of the project required intensive study of the Tenth Symphony sketches to determine what Beethoven's intentions were for the piece. Using the composer's completed works as a template, they puzzled out which of Beethoven's incomplete musical phrases belonged in which of the four movements of the symphony.
The AI's task was to figure out how to fill in the gaps. It needed to take very short musical phrases of just a few notes and expand them into longer, more elaborate structures. It did this by learning how Beethoven built his Fifth Symphony on a simple four-note motif. It also needed to understand the musical form of each expanded phrase—scherzo, trio, or fugue—to ensure it went into the correct movement.
As the project went on, the computer scientists found that the AI would have to be responsible for a lot more.
"The to-do list grew: We had to teach the AI how to take a melodic line and harmonize it," said Elgammal. "The AI needed to learn how to bridge two sections of music together. And we realized the AI had to be able to compose a coda, which is a segment that brings a section of a piece of music to its conclusion."
An example of one of Beethovens sketches. Image credit: Beethoven House Museum, CC BY-SA
It also had to determine what instruments would play which parts, and it had to do all of this in a way that the great composer may have done it.
The team held its first test in front of journalists and, more importantly, music scholars and Beethoven experts. It asked them to see if they could figure out where the composer's work ended, and the AI continued. The music experts were stumped and unable to tell which parts were composed by the AI.
Elgammal performed a second test in front of experts, some of whom were familiar with Beethoven's Tenth Symphony sketches. Only those who had "intimate knowledge" of the incomplete works were able to pick out the AI-constructed parts. The test was enough to conclude that the algorithms had done a good enough job completing Beethoven's Tenth.
Beethoven's Tenth Symphony will be released to the public on October 10 after its debut performance in Bonn, Germany. You can get a taste of how well the AI constructed the piece from this short selection provided by Smithsonian Magazine.
Image credit: Perrant (CC BY-SA 3.0)
