In context: Researchers at OpenAI have created a team of bots that can beat the top one percent of amateur Dota 2 players. The research is an extension of the AI they developed last year that could defeat the some of the best professional players in 1v1 matches.
This time around multiple bots come together to work cooperatively in 5v5 Dota faceoffs against human teams. This is significantly different from the previous challenge of 1v1 because multiplayer matches require a considerable amount of teamwork, coordination and long-term planning.
So far OpenAI’s team of bots have been able to beat all of the top amateur teams it has faced, including one that was led by pro Dota player William “Blitz” Lee.
According to Lee, in one of the matches, OpenAI Five made “one of the highest level plays that you can make” by owning and controlling two-thirds of the map. He was even more stunned to see the bots perform the feat in two consecutive matches.
“The ability to intuitively do this is insane,” said Blitz. “Doing it one game I could maybe chalk it up to just dumb luck. Doing it two games in a row, flipping the sides means that it’s more than just coincidence.”
OpenAI says that the bots were not taught this strategy like a coach would train a human team. Instead, the artificial team worked together playing against itself a multitude of times on over 100,000 CPUs. Using reinforcement learning, OpenAI Five effectively trained itself the equivalent of 180 years per day and has thousands of years of experience behind it.
It is no wonder that the AI team was able to come up with such top-level strategies on its own with that amount of training.
“It was pretty easy to quantify for me,” said Lee about the bots’ use of advanced strategy. “It took about eight years for me to learn the strategies that the bot was intuitively doing.”
As good as the computerized Dota 2 team is, OpenAI says that it is still a long way from beating professional Dota teams. However, the OpenAI engineers are up for the challenge and are already at work on the next version of the bot. It will be hosting a live match in July just before the Dota World Championships in August. There the researchers will pit the bots against a team of top players to see how it performs.
The researchers hope that the advancements they are making in machine learning and reinforcement learning will lead to constructing bots that can tackle complex real-world situations such as coordinating a metro system.