Notifications
COMPLETED
Creative Application Award
Article
SpaceY
Updated 8 months ago
1.6 K
0
SpaceY
This project demonstrates how machine learning algorithm can be used to teach a rocket to take off from one planet, then fly and land on another one.
The following video demonstrates the final result (builds are attached at the end of this article):

Simulation Description

  • Game area is a rectangular area that contains two planets - Planet 1 and Planet 2.
  • Planet 1 is randomly positioned in a small central part of the game area, Planet 2 is placed randomly in the remaining space.
  • Planets have gravity fields, they are denoted by circles around the planets.
  • At the beginning of each session rocket is located on Planet 1.
  • Rocket has six engines that apply different accelerations (they are denoted by red arrows in the picture below) at different positions.

Success \ Goal

The goal is to take off the rocket from Planet 1 and to land it on Planet 2. Landing is considered successful when both landing legs touch Planet 2.

Failures

The following situations are considered as failures:
  • Rocket hits any of the planets with its hull.
  • Rocket moves out of the game area.
  • Rocket starts flying too fast (faster then some predefined constant). This condition is applied to ML-Agent simulation, when rocket is controlled manually by a player this condition is removed.
  • Rocket starts rotating to fast (faster then some predefined constant). This condition is also applied only to ML-Agent simulation, when rocket is controlled manually by a player this condition is also removed.

Training

ML-Agent's brain uses discrete action space type - 6 actions are used for different engines, and one action is a "do nothing" action.
The training was performed in a few separate steps:
  • The model was trained on a medium size field without curriculum learning until it reached a good enough success rate.
  • Then the training was continued using curriculum learning - on each step the size of the game area was increased until all curriculum learning lessons were completed.
  • After that the training was continued with modified Hyperparameters ("beta" was reduced).
After these training sessions were conducted, the algorithm was able to achieve ~91-93% success rate on a large game field.

Manual mode

Simulation can be switched to manual control mode by pressing "Brain: ML-Agent" button at the top of the screen. In the manual mode use the following keys to control the rocket:
  • "Q" - upper left side engine;
  • "E" - upper right side engine;
  • "A" - bottom left side engine;
  • "D" - bottom right side engine;
  • "S" - main bottom engine;
  • "W" - top stop engine.

Conclusion

The training was performed on an old mobile Geforce GTX 460M graphics card and it was rather slow. It is obvious that the above-described training process can be improved in many ways.
Unity Machine Learning tools open wide opportunities for game developers.

References

  • Source code
  • Demo video
You can download Windows and Mac builds here:
  • SpaceY_Windows_ver.0.03.zip
  • SpaceY_Mac_ver.0.03.app.zip

Mikhail Bykhovskiy
1
Comments