[ Part 1 ]How to setup TensorFlow ML library with Unity Engine for developing Machine Learning based apps/games.
Published 12 days ago
[ Part 1 ]How to setup TensorFlow ML library with Unity Engine for developing Machine Learning based apps/games.
In this article I will show you how to setup the Google's TensorFlow ML library with the popular Unity game engine. Since the setting up is somewhat tedious for many, I have tried my best to cover all the steps.
This setup process can be broken down into 2 parts namely GUI part and Command line part.
GUI part installations are pretty much straight forward. But in order to setup the environment successfully, you should have a basic understanding of the following tools and their specific commands relevant to this article(mentioned in brackets),
  1. Python language (pip/pip3)
  2. Anaconda (conda, activate)
I will cover the commandline part in another post to keep this article short, concise and also to help out non-unity developers who are only interested in TensorFlow setup.
Lets start with the GUI part.
NOTE : Close all graphics applications , Unity Instances and also visual studio IDE before installing since it may interfere with Nvidia driver installation.
CUDA related tools setup:
1. First download the CUDA Toolkit 9.0 from the below link, You may also try latest version if you wish. Select the correct OS, Architecture, Version. For Installer Type, choose exe(local). Its a big file(around 1gb).
2. Once downloaded, Install the Nvidia CUDA toolkit with the express option as shown below,
3. Then, download the NVIDIA CUDA Deep Neural Network library (cuDNN) This is a GPU accelerated neural network lib used by TensorFlow to train your game entities. You must have a Nvidia developers account to download it. Once successfully logged in, you should see a download page similar to this,
Since we have downloaded CUDA Toolkit 9.0 in step 1, we should download cuDNN for CUDA 9.0. This is an zip .
4. Open the zip file and drag&drop all the folders to your CUDA Toolkit installation dir. Your CUDA toolkit install path will be similar to ,
[drive]:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0
This is how it looks in my PC.
Once the folders are replaced in the CUDA toolkit dir. We need to add these folder's path in system Environment variables.
5. Open the System Properties window by typing "edit environment" in start menu or Another shortcut way is to type SystemPropertiesAdvanced in run(Windows + R) window and hit Enter.
NOTE : Make sure you specify the path where you installed on your system. Below path are for reference only.
Add your following paths to the System variable's Path .
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\lib\x64
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\extras\CUPTI\libx64
Then create a new System Variable CUDA_HOME and enter the path
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0
Finally , click OK and close the System Properties window.
Now we have done setting up CUDA related tools. Next we will setup the python environment where the Tensorflow library handles ML processings.
TensorFlow & Python environment setup:
1. First, download and install Anaconda from below link,
NOTE : Its better to remove other standalone python versions since Anaconda handles multiple python versions more efficiently.
2. After installation, goto Start menu , type Anaconda Prompt and hit Enter. You should see command line window. Here , we will use conda command to create a python environment for managing Tensorflow packages. If you are doing it in standalone python , you may face package corruptions and other version incompatibilities and that is why we will use conda to isolate things.
This is command format to create a conda environment:
conda create -n {your env name} python=3.5.2
I ran like this,
conda create -n unityml python=3.5.2
When executed, it will ask for confirmation to install some dependencies . Press 'y' and continue with it.
This is how it looks when things go well,
Here, unityml is our python environment name.
3 . Once created , we should activate it in order to use it . To activate ,
conda activate unityml
This is what I got,
Also, you may see that (base) is now changed to (unityml). We are no more working in base python environment. conda gave us a separate python environment to play with. Always make sure you use this same environment in future.
4. Now we will install tensorflow library into our unityml python environment.
pip install tensorflow-gpu
Once installed, type python to enter the python interpreter mode. In that, type the following to check whether the tensorflow package is installed properly,
import tensorflow as tf
If no error/message is printed , then we have successfully installed it. Else check for syntax mistakes or refer this official link :
And so, we have done setting up CUDA toolkit, CUDA Deep neural network libs and finally a separate python environment with TensorFlow ML library . You can go to the official website for trying out some sample ML projects.
In part 2, we will see on how to use this newly created unityml py-environment to train our game entities in Unity Engine.
Thanks for reading.
References :

Praveen kumar
Senior Unity Developer with AR, VR skills. - Programmer