A new unity learner wants to learn more about ml-agent
I am new to unity and I have developed Space Shooter Tutorial. Then I introduced to ml-agent extension with the guidance of Prof. Sam Devlin and tried to use it within the game. This is the main version of Space Shooter Tutorial:
As you can see in the video above, the spaceship tilts at the time of going left and right. Moreover it can move upward and downward. I found out I had to simplify the environment to be easier to learn. First, I simplified spaceship's control by limiting its movement just to left and right. Moreover, I disabled the tilt and acceleration of movement to become an old fashion movement control. Second the environment is simplified by reducing the number of asteroids in each wave to just one. So the result is shown in video below:
Now it is ready to be learned. 3 state variables are defined:
myState (discrete) - corresponding values:
on the left side of enemy (on the x axis)
on the right side of enemy (on the x axis)
enemyState (discrete):I have defined a threshold on z axis for asteroid position
Killable position (does not reach to threshold)
Too low to kill (passed the threshold)
distanceState: relative distance between the agent and the asteroid on x axis (continues)
There are four actions:
I have tried different methods for rewarding. Now I reward the agent when it shoots while he is below the asteroid. However, before that I tried small rewards to encourage the agent to come below the asteroid. In final version, I removed them, because I could not get much better results by them.
Here you can see the result of training in the video below:
And I have used PPO code provided in ml-agent package for training.
I would be grateful if you share your knowledge through comments about how it could be better.