Article
Maze Solver&Blocker with ML
Published a year ago
413
3
Maze Solver&Blocker using Curriculum

Short Intro

My attempt at training a maze solver agent with machine learning using curriculum learning to train both the solver and blocker (units preventing solver agent to finish). The aim of the solver agent (teal) is to avoid hitting any walls or blocker agents (orange) and get to the finish line (red). This is the end result:

How it works

Maze solving algorithm is independent of any maze shape as it uses several raycasts to determine what's around the agent and give rewards according to them. I've mostly used positive rewards and also tried to reinforce perfect behaviour by not using discrete rewards (e.g reward for staying in middle gradually decreases as the agent gets closer to sides).

After getting the solver agent to find its way on a straight path, I've built a maze and placed 5 different points to be used as finish line. Using a curriculum, I advanced the finish line after a certain point was reached (normally required point was somewhere between a perfect run and a sloppy run).

When the solver agent learnt to navigate an empty maze perfectly, I added another brain for the blockers and used the trainee solver agent to train blockers on the same maze. With no prior knowledge of moving objects in course, the solver was pretty much running in a straight line. Blockers were rewarded for getting close to solver when it's in a certain range and eventually hitting it (hence reseting the course).

Finally, when the blockers were perfectly blocking the solver, I switched the training roles one last time and started training the solver against the previously trained blockers. Due to narrow walls of the maze, I had to keep the blockers slower than the runner so it had some room to evade.

Progress!

Being an absolute beginner to machine learning, I kept videos of all my silly attempts (especially first two attemps). It took me one month to figure out how everything worked but I really enjoyed the experience.

P.S: I've written a simple script to build academy for training (changing the brain to external) and switching back(changing brain to internal and assigning .bytes file). Feel free to use it: https://github.com/oayhan/MLBuilder

Omer Ayhan
Senior Game Developer - Programmer
5