2nd Prize
Metal Warfare - Real Time Strategy game
Updated 3 years ago
11.7 K
Metal Warfare - Real Time Strategy game, special edition for AI & ML challenging
Metal Warfare is an RTS game I designed for both entertainment and education purpose. The education will be focused on teaching player how to build their own Artificial Intelligence (AI) and challenge online with other player or AI. The final milestone of the game will let player using Machine Learning (ML) to train their AI.
As the release of Unity ML-Agents, I will have some showcases for you on how to do Machine Learning for your AI on an real time strategy game.
The game is built from simple to complex, at very first levels on the game, player will have a Base, attack enemy by "Missile Launcher" and defense by "Anti-Air":

Let have a look at the game level we selected to do training:
After checking the Unity ML-agents example and our game level, we find out that 3 things we need to add into the training model:
1. The Idle action: Almost the time, player need to wait for enough power to do the action. There is no sense we need an action for those moments and we do not need any learning in that case also.
2. Action filtering: Let say at position (1,1) on the map, you already have a building, so the action to build another building on that position should be invalid, we should filter it out. Of course, you can think of letting ML study it. But just think back, we have many other interesting parts on the game need the ML to learn right?
3. Delay rewards for the action: We have action launch a missile, we only know the result of that only when the missile come to the destination. Let make the learning smarter, we will store the action and update the rewards when the missile done his job.
Our learning model will look like this:

The states array will content 5 parts:
Action ID - rewards for previous action if any - is Idle or not - available action - learning states
The learning states will be separated and joined with the observation of the mini map for learning, Other parts of the states will be used for filtering out the actions.

The rewards for the game we decided will be: Lose the game: -1.0f Lose 1 building: -0.01f Win the game: +1.0f Plan missile to building: +0.0003f Plan missile to fog: +0.00015f Anti defense success 1 missile: +0.0001f Successfully clear 1 fog: +0.001f Successfully cause damage to an enemy building: +0.002f Successfully destroy 1 building: +0.008f (20% rewards will be account into the launcher building action)
This is how the game was played after some training:

On the training process, we find out an epic situation that the ML can learn: There is a time when ML nearly lose, only have 1 launcher when enemy has 1 anti and 3 launchers, but the ML notice that the enemy HQ only have 1 HP remain. It choose to put that final missile direct to enemy HQ and expect the 10% that anti will miss. Luckily, it work! here is the victory screenshot:

Ok, now let have a look at our normal game play:

So now we have the marine, the AOE missile,... and more. How the training could be? Is it amazing if the marine know how to go out of turret range and attack other building, and when there are enough number of marines, they will come together to take down the turret?
Here is my shared GitHub:
Thank you for reading my article. Hope you enjoy.
Chau Chi Thien
Chau Chi Thien
a year ago
Great Robert, hope you will success!
Robert Futch
a year ago
Great application of the tech! I'm working on a strategy related ML game as well!
Chau Chi Thien
2 years ago
Hi Osama, we trying to make the game to that state where you can experience ML on it. But unfortunately, we do not have budget for the project, so we do not know when it will be available.
2 years ago
can we download the game and experiment with different algorithms in unity?
Chau Chi Thien
2 years ago
DongA great game which have great AI!
Hello Dong, the game is available on the open AI and ML is on plan (we still waiting for enough buget for kick off)