This is the first project in the Udacity Deep Reinforcement Learning Nanodegree. It requires students to develop and train a Deep Q-Network (DQN) model to collect yellow bananas in a simulator.
The Agent is incentivised to find Yellow bananas with +1 reward and disincentivised to find Blue bananas with -1 reward within the simulator.
The simulation contains a single agent that navigates a large environment. At each time step, it has four actions at its disposal:
0- walk forward1- walk backward2- turn left3- turn right
The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana.
The environment is considered solved when the average reward (over the last 100 episodes) is at least +13.
It is recommended to follow the Udacity DRL ND dependencies instructions here
This project utilises Unity ML-Agents, NumPy and PyTorch
A prebuilt simulator is required in be installed. You need only select the environment that matches your operating system:
Linux: click here Mac OSX: click here Windows (32-bit): click here Windows (64-bit): click here
The file needs to placed in the root directory of the repository and unzipped.
Next, before starting the environment utilising the corresponding prebuilt app from Udacity _Before running the code cell in the notebbok, change the file_name parameter to match the location of the Unity environment that you downloaded.
- Mac:
"path/to/Banana.app" - Windows (x86):
"path/to/Banana_Windows_x86/Banana.exe" - Windows (x86_64):
"path/to/Banana_Windows_x86_64/Banana.exe" - Linux (x86):
"path/to/Banana_Linux/Banana.x86" - Linux (x86_64):
"path/to/Banana_Linux/Banana.x86_64" - Linux (x86, headless):
"path/to/Banana_Linux_NoVis/Banana.x86" - Linux (x86_64, headless):
"path/to/Banana_Linux_NoVis/Banana.x86_64"
For instance, if you are using a Mac, then you downloaded Banana.app. If this file is in the same folder as the notebook, then the line below should appear as follows:
env = UnityEnvironment(file_name="Banana.app")
Then run the Navigation.ipynb notebook using the drlnd kernel to train the DQN agent.
Once trained the model weights will be saved in the same directory in the file checkpoint.pth.
The model weights are used by the Trained Agent.ipynb notebook against the simulator.
