CS 370 final project — pirate agent using deep Q-learning
This repository contains my final project for CS 370 at SNHU. The project is a pirate intelligent agent that learns to navigate an 8x8 maze and find the treasure using deep Q-learning.
I was given starter code that set up most of the environment for me. TreasureMaze.py defined the maze itself, the rewards, and the rules for how the pirate moves around. GameExperience.py handled the experience replay memory and the Bellman updates that the Q-network uses to learn. I was also given the neural network in build_model(), the train_step() helper that runs the gradient updates, and the play_game() and completion_check() functions that test the trained agent.
My job was to complete the qtrain() function, which is the training loop that actually teaches the agent. I wrote the logic that picks an action using epsilon-greedy, takes the move in the environment, stores the transition in memory, and calls the training step on every move so the Q-values actually converge. I also added a target network that gets synced periodically so the Bellman targets stay stable, and I made sure to store the done flag as a proper True/False instead of the raw game status string so the replay buffer would bootstrap future rewards correctly. After tuning the hyperparameters, the agent reached a 100% win rate and passed the completion check from every free cell at epoch 278.
What do computer scientists do, and why does it matter? Computer scientists solve problems using code and systems. That can mean building apps, analyzing data, training models like this one, securing networks, or designing algorithms that nobody has thought of yet. It matters because almost every part of modern life runs on software now — healthcare, transportation, finance, education — and the people who build those systems shape how well they work and who they work for.
How do I approach a problem as a computer scientist? I try to break the problem down into the smallest pieces I actually understand before writing anything. For this project, that meant reading the starter code first, figuring out exactly what the maze expected, what the replay buffer expected, and only then writing the training loop. When my first version didn't learn, I didn't just change random things — I looked at the output, figured out that the network was never getting trained often enough, and fixed that one thing. When the fixed version trained but still failed the completion check, I traced through exactly what the greedy policy was doing from the weak starting cells and found that the mask during training was hiding the problem. Each fix was targeted at a specific bug, not a guess.
What are my ethical responsibilities to the end user and the organization? Even on a small project like a game agent, I'm responsible for writing code that works the way I say it works. If I claim the agent reaches the treasure, it actually has to reach the treasure — not just during training but in real gameplay. On bigger projects, that responsibility scales up fast. If I work on something that touches real people's data, safety, money, or decisions, I have to be honest about what the system can and can't do, protect the data I'm trusted with, and flag problems when I see them instead of hiding them. I also have to respect the organization I work for by writing code that other people can read, maintain, and build on after I'm gone.