Asteroid Buster Space AI
Published 2 years ago
An AI trained to play a classic gameplay style.
I created an AI to test out Unity's new ML-Agents and decided to test how well I could get it to play a game I released on android a few years ago, just to find out if it could do it.
To train the AI, first the game had to be simplified, by removing power-ups, and making it so that a single crash into an asteroid resulted in the agent restarting from the beginning. This meant the AI had to learn to survive using only the basic controls and starting weapon in the game. Once the AI managed to get the basics down, things get more complicated, and it had to learn to find power-ups, and make it through many waves on a limited number of lives.
The AI gets data to tell it where the ship is in the world, its rotation, and the world position of asteroids relative to the ships position, and later in training, the position of power-ups that have been dropped by the asteroids it has destroyed.
As the AI plays, it is rewarded for getting points, and punished for deaths. It also receives a reward when it completes a wave, or collects a power-up.
At first the AI was unable to make it beyond the first wave on most attempts, and would explode and be destroyed almost right away. But after about a 20 minute training at 3x the speed a normal person plays, the AI was able to start developing a strategy, and could make it a couple waves in usually before crashing and restarting.
After re-enabling power-ups and allowing the agent to keep playing after he gets destroyed, as long as there are lives left, the AI began to show real potential as an AI player. There was emergent behavior where the AI would prefer to go collect power-ups before taking out the last asteroids many times.
One strategy that emerged is where the AI spins around fast in a circle to try and blow up asteroids as quickly as it can, without getting too close. I refer to this as the "spray and pray" strategy:
At the end of testing, the AI had been trained in a variety of ways, upwards of 50 million steps (across several different techniques) which took sometimes over 12 hours of training at high speed. While the original goal was to train the AI to become close to, if not better than a human player, he fell short, only maxing out at a little under fifty thousand points in one game, and the best score a human had ever posted to the leaderboards was over 400 thousand points!
So while this AI may not be the best player to ever play this game, he is certainly interesting to watch, and plays in strange ways.
Perhaps with a perfect training setup, and a nice long time for him to practice, he might eventually become much better!
Developer - Programmer
Victor Hernandez
2 years ago
Looks like the agent lacks info. try giving it the velocity components for the asteroids and itself.
Mehdi Ourabi
2 years ago
Interesting to see the spray and pray strategy emerged. One could have guessed that a computer would be better at aiming and thus should rely more on its skills than a human ?But maybe the AI brain lacks some State to pre-shoot asteroid position...