Game test with ML Agent
Published 7 months ago
Using Machine Learning to test game play.
If you have been using Unity's ML agent, you should already have some idea about its capabilities. This article will not go into the detail of the development or show you how to train your agent, but to highlight the issue that I encountered while prototyping a party game and how ML agent can be helpful in this situation.

Need a hand, a couple more perhaps

It's never easy to find someone to test a game with you, especially during the prototype phase and you need to simulate a number of test cases. I can't just make team members as dedicated game testers when they have other important tasks to work. Testing a party game alone is plain silly. Conventionally, I would make AI behaviour script from scratch just so I can test multiple players scenario. Usually the AI behaviour would be rigid since all it does is just repeating the same action.
Unity's Machine Learning agent has the potential in my case, because it adapts to the environment autonomously and requires minimal scripting to get the result I need.

The task

The task is to get the agent to learn the game with minimal supervision, understand the game rule and display decision making characteristic. For instance, collecting coins in the game will earn the agent points and avoid the fire wall. In this context, I'll be using Reinforce Learning method to train the agent. The idea is to reward the agent if it is doing the right thing and vice versa. Over time or after certain amount of steps, the agent will learn the most efficient move to acquire the target, and apply the right amount of force to move around the environment so it will not get into the fire wall, which will kill the agent.
Another advantage of using Machine Learning is I can use multiple agents to accelerate the learning process, and I can also do it with multiple environments in one training session. Each agent will have different characteristics such as different amount of health point, turning radius and weapon firing rate. This is to ensure the agents are able to learn and perform under different settings and scenarios.

Let the training begin

Once I'm happy with the training prep, I start the training session. I let the training runs for 5,000,000 steps. Once the training has completed, the constructed model is imported back to Unity and is tested.


The agents worked exactly as expected when I apply the model to the test. Although not perfect, the agents collected the coins as it should and one thing that I noticed was, occasionally the agents will start to compete for coin. This is perfect as it will give me some sense competition as the agent tries to rob the coins from me. The agents did found a few flaws in my level design, which is a plus.
I definitely can see how Machine Learning can be used to test my game up until the point when human is required to test the game in the later phase. Machine Learning has demonstrated that I don't need complex AI behaviour script to get similar result.

Danny Chow
Senior Game Programmer - Programmer