Music scholars and computer scientists completed Beethoven's Tenth Symphony aided by machine...

Cal Jeffrey

Posts: 3,217   +883
Staff member
What just happened? Artificial intelligence has completed Beethoven's Tenth Symphony—or at least, what that piece of music could have sounded like had he finished it. A world premiere of the music will take place on October 9, 2021.

In 1817, the Royal Philharmonic Society in London commissioned Ludwig Von Beethoven to compose his Ninth and Tenth symphonies. He completed the Ninth Symphony, which concludes the fourth movement with the all too familiar "Ode to Joy" (below). Due to declining health and his subsequent death in 1827, he never completed his Tenth Symphony. All that remains of that work is some scrawled musical sketches.

The Karajan Institute partnered with a group of scientists from AI startup Playform led by Ahmed Elgammal. The team spent two years training an AI model using Beethoven's complete works, the Tenth Symphony sketches he left behind, and what is known about his methods of composing music.

Elgammal tapped Walter Werzowa, the mind behind Intel's bong jingle, to merge what Beethoven left behind with the AI-constructed music. Computational music expert Mark Gotham led efforts to transcribe the centuries-old sketches and process Beethoven's entire work to train machine learning algorithms. Harvard musicologist Robert Levin, who had previously completed several musical compositions from Mozart and Bach, also helped on the project.

"In a large room with a piano, a blackboard and a stack of Beethoven's sketchbooks spanning most of his known works, we talked about how fragments could be turned into a complete piece of music and how AI could help solve this puzzle, while still remaining faithful to Beethoven's process and vision," Elgammal said.

The human side of the project required intensive study of the Tenth Symphony sketches to determine what Beethoven's intentions were for the piece. Using the composer's completed works as a template, they puzzled out which of Beethoven's incomplete musical phrases belonged in which of the four movements of the symphony.

The AI's task was to figure out how to fill in the gaps. It needed to take very short musical phrases of just a few notes and expand them into longer, more elaborate structures. It did this by learning how Beethoven built his Fifth Symphony on a simple four-note motif. It also needed to understand the musical form of each expanded phrase—scherzo, trio, or fugue—to ensure it went into the correct movement.

As the project went on, the computer scientists found that the AI would have to be responsible for a lot more.

"The to-do list grew: We had to teach the AI how to take a melodic line and harmonize it," said Elgammal. "The AI needed to learn how to bridge two sections of music together. And we realized the AI had to be able to compose a coda, which is a segment that brings a section of a piece of music to its conclusion."

It also had to determine what instruments would play which parts, and it had to do all of this in a way that the great composer may have done it.

The team held its first test in front of journalists and, more importantly, music scholars and Beethoven experts. It asked them to see if they could figure out where the composer's work ended, and the AI continued. The music experts were stumped and unable to tell which parts were composed by the AI.

Elgammal performed a second test in front of experts, some of whom were familiar with Beethoven's Tenth Symphony sketches. Only those who had "intimate knowledge" of the incomplete works were able to pick out the AI-constructed parts. The test was enough to conclude that the algorithms had done a good enough job completing Beethoven's Tenth.

Beethoven's Tenth Symphony will be released to the public on October 10 after its debut performance in Bonn, Germany. You can get a taste of how well the AI constructed the piece from this short selection provided by Smithsonian Magazine.

Image credit: Perrant (CC BY-SA 3.0)

Permalink to story.

 

tellmewhy

Posts: 122   +59
Very nice, so this means that if they give some notes of a known complete symphony the neural network it will produce something close to the original composition?? :yum
 

Cal Jeffrey

Posts: 3,217   +883
Staff member
Very nice, so this means that if they give some notes of a known complete symphony the neural network it will produce something close to the original composition?? :yum
Well it took a lot of human work too. Plus it was only trained on Beethoven. If they wanted to do it again with a different composer I imagine they would have to repeat the same process. Elgammal didn't go into super great detail about how the AI worked, but it did sound like it was pretty much specialized for this specific task. I imagine with more effort and tweaking they could probably develop it to the point where it could do the same with any composer so long as they feed it enough of his or her work.

Did you listen to the snippet? It sounds really good. I interested in hearing the CG Tenth in its entirety.
 

tellmewhy

Posts: 122   +59
Well it took a lot of human work too. Plus it was only trained on Beethoven. If they wanted to do it again with a different composer I imagine they would have to repeat the same process. Elgammal didn't go into super great detail about how the AI worked, but it did sound like it was pretty much specialized for this specific task. I imagine with more effort and tweaking they could probably develop it to the point where it could do the same with any composer so long as they feed it enough of his or her work.

Did you listen to the snippet? It sounds really good. I interested in hearing the CG Tenth in its entirety.
I mean if they give it as input music notes from a finished Beethoven's symphony (ex the 7th) what the propabilities that it will produce the same thing or something close to it and what the propabilities that it will produce something completely different?

The snipped they show it is nice but I think is like the Deep Blue win against the Kasparov, which had a team of grand masters behind the computer to consantly help it to adapt. Music composition it will be tougher than chess play because it’s not deterministic like chess and the neural network can’t easy adapt and wrap around a chaotic field.
 

Cal Jeffrey

Posts: 3,217   +883
Staff member
I mean if they give it as input music notes from a finished Beethoven's symphony (ex the 7th) what the propabilities that it will produce the same thing or something close to it and what the propabilities that it will produce something completely different?

The snipped they show it is nice but I think is like the Deep Blue win against the Kasparov, which had a team of grand masters behind the computer to consantly help it to adapt. Music composition it will be tougher than chess play because it’s not deterministic like chess and the neural network can’t easy adapt and wrap around a chaotic field.
Yeah, as I understand it it was a bit different than that. Essentially, the human music experts were used to interpret not just where Beethoven's sketches belonged in the symphony, but what his intentions were with the piece. Highly subjective to human opinion, but if you are going to do that, then who best to provide input other than musical scholars and Beethoven experts?

The AI's job was to basically fill in the blanks left (among a multitude of other complex tasks) that the scholars couldn't figure out. The computer had to learn how Beethoven built his music by learning how he built other works like the Fifth Symphony. It was trained with both the completed pieces AND Beethoven's notes and sketches of those completed Symphonies.

Obviously, nobody will ever know how close the AI got to what he was really trying to create, and another skilled composer like Robert Levin, who had finished a number of incomplete works by Bach and Mozart, might have come up with something else. That said, Levin did help on the project so the Tenth was likely in such an early phase that the task was even too monumental for him to attempt.

EDIT: Also, I believe I did see in passing that other composers had tried to complete the Tenth, but I did not have time to listen to their results.