During August 7-12, one of the largest tournaments in online gaming was held. The International 2017 was the seventh edition of an eSports championship for the popular online game Dota 2. Top players from all around the world competed for their share of the 24 million dollar prize pool. The tournament was attended by 20.000 fans and watched by hundreds of thousands online via Twitch.tv. The game is usually played in two teams of 5 players, and the winning team took home $10,862,683.
This year, at The International 2017, OpenAI* showed off their confidence in their machine-learning artificial intelligence software by creating a bot designed to play the game Dota 2 in a one versus one format. They gave the bot some very basic understanding of the game, receiving some feedback on what is ‘good’ and ‘bad’ in the game, after which the bot was released to practice and self-teach the game. Interestingly, the programmers didn’t have to give a lot of rules to the bot, because it can keep improving itself as it plays and practices against itself. Within a month, the bot went from laughably bad to achieving a superhuman skill level.
OpenAI then showcased the bot by letting it play against the top professional players, players who have put in more than 10,000 hours of practice in this particular game and play this game for a living. Not surprisingly, the bot won 26 matches against top players and even a world champion and lost only 2 times. The players admitted in an interview that the bot did things they never even thought of, but which they now also will incorporate in their own game since the strategies work so well. In this way, the bot became better than any human player, so pros now actually want to use the bot as a training partner to take their gaming skills to the next level.
It is very interesting to see how fast artificial intelligence can surpass human skill levels in certain settings with only minimal inputs. To see that after a few weeks, the bot reached a skill level which left the professional players in awe is really amazing. OpenAI hopes to keep improving the bot so they can eventually create a team of 5 bots which could play against pros or against each other. Of course, one could say that this is only a game, but I think that the OpenAI team did something very cool here, as they reached millions of people with this project, who might also get interested in AI and some of them might even discover applications for which AI and machine learning could be useful in our personal lives. There seems to be a great amount of possibilities within this field and I am excited to see the developments within AI over the next few years. As OpenAI say it:
“This is a step towards building AI systems which accomplish well-defined goals in messy, complicated situations involving real humans.”
*OpenAI (https://openai.com/) is a non-profit research start-up co-created by Elon Musk which has the goal of furthering the path to safe artificial intelligence. By developing and promoting safe artificial intelligence, they hope to benefit humanity as a whole. All of its patents and research are available to the public. If you are interested (and have some programming knowledge) you can even download free software from OpenAI’s website and create your own learning algorithm for any application you might think of.
Interesting post Rogier! The fact that pro players are actually enthousiastic about the bot’s playing style and want to improve their own gameplay based on this experience is very cool! This makes me wonder if real life athletes can also improve their performances based on watching bots simulate everyday sport situations such as corner kicks or running marathons. What is your opinion on this?
Thanks for your comment! I think AI would be very much be able to train athletes in these sports, but the problem would be finding a way to have a real life robot being able to make the same movements as human players would. I found an example of how another AI bot (ISAAC) was able to teach itself two sports, hockey and golf, and is now able to consistently score goals / make puts. This is a simple example, but I’m sure in a few years it could be more advanced. I think the benefit with a game over a real life sport is simply that no additional hardware is required for the bot to start training, otherwise it should definitely be possible, for example, to create a leg which can consistently score corner kicks in football.
Here is the video which explains and gives a video representation of how Nvidia got their bot to learn golf and hockey, you might find the presentation interesting:
https://youtu.be/oa__wkSmWUw?t=7m4s