What's gonna work? AI Teamwork!
Published a year ago
My first experience with the world of multi agent machine learning


As a highschool student and the future CTO for SKYNET (jk),I feel no AI takeover of the world is complete without autonomous agents that can coordinate complex procedures. The inspiration for this project nether Terminator nor Japanese wrestling, but the locomotive industry. After touring my local BNSF train station a few years ago I could help but notice that most of the workers had just been laid-off and replaced by hard coded algorithms. One such program is positive train control, meaning that we have self driving trains now. What will they think of next; Self driving elevators? Moreover, the few human workers that were still there were little more than a formality. For example, by law there needs to be a conductor in every autonomous train. In this instance the person in the driver seat is little more than a human cutout . Unfortunately, this example is just a canary in a coal mine. (which is ironic because even a Canary's job in the mining industry was taken by an air quality sensor long ago )


My setup consists of multiple multiple agents with the same brain and inverse reward functions for each team. The demonstration above is a mixture of multiple brains. I used the unity ML kit along with the CUDA ML library, and Tensorflow for my genetic learning algorithm. I did all my training in a Jupyter environment with all the standard settings .

A light at Training in the Tunnel

First I started with 1 v 1 cube matches in the traditional Sumo style, however I looked into the how to make a team match of cubes. I trained 3 brains with different priorities. For the first cube I trained it individually with a fitness function with an incentive to stay in the arena. Next, I trained a cube that would collide with a cube on the opposite team. Finally, I trained the last brain in an environment that had multiple agents on both teams. In summary, I have 3 brain instances each with a slightly different goal.

Testing the waters

If there is one thing nuanced about machine learning is the increased chance of failure. Now, in other disciplines of science, If half of your experiments fail, fine,science improves, you as a scientist still get published. On the other hand as an engineer, if half of your bridges collapse, You'll not only get headlines, but more importantly you'll end up in jail. On that note, putting the results of each instance of training was a unique challenge for me. I had to face the fact that my artificial intelligence acting dumb reflects less on my ability, but random chance. One can even say there is something spiritual in putting faith in an algorithm. Anyway, the main challenge was getting the cubes to recognize each other's inverted reward function. Fortunately after the 10000 th step things usually improved. The pace of improvement was slow however the end result was visually more spectacular.


When It comes to Machine learning at this point there is much human knowledge to acquire. However, I see it as a privilege to get to work with the very same algorithms that may one day replace me. It is also quite interesting to live in a time when work as we know it will undergo a paradigm shift. Automation will therefore allow us inferior humans to do work that fulfills us instead of toil in monotony.

The Github link is here.
Marek Ferguson
Digital Learning Specialist Crosstown High - Student