This is a simple spaceship racing game based on the ML-agent.I have made a racing game in which a space ship travels from the beginning to the end in a random-generated racing track.Player can control the space ship using the arrow keys when the BrainType is Player.I haved train the AI with the machine learning agent, and the space ship managed to travel from the beginning to the end using the using the trained AI model.
github : https://github.com/rockgong/ml-agents
In the github repo, my project is in the branch named "challenge".
The space ship can do two actions : boosting and turning.And the AI can make the space do these two actions and it can also detect the track around the ship.The result of the detecting will be passed to the AI in CollectState method in my agent.Using the detecting state the AI can decide whether the space ship is facing to the right or wrong direction, and make the decision which of the two actions should be applied to the spaceship.If the space ship is facing the right direction, doing boosting action comes up with a positive reward. If the space ship detect a better facing direction on the left, the turning left action comes up with a positive reward and the turning right action comes up with a negative reward.If the space is facing a totally wrong direction, the boosting action can come up with a negative reward.
After a 2,000,000 steps training.The space ship can completely travel the whole trace.The racing track is random-generated by codes.By tuning the parameters the track-generating-code can generate different kind of track.So the AI can generally make the space ship travel completely in many different kind of track.
I also use curriculum in the training process.Because the rewards and the states is all just in time, the length of the track will not make any difference in the difficulty of the task.So I tried to tune the difficulty by tuning the width of the track.If the track is the wider, the more chance that the space ship can face to the right direction and the less change the space ship hit the wall.At the beginning of the training, the wide of the road is 8.0 (unit of unity scene).After a while of training, the reward can reach the threshold many times, and the track become narrower and narrower, finally the width becomes 4.0.