ForagerRL_step11

11. Testing in GAMA GUI

By Killian Trouillet

---

Step 11: Testing in GAMA GUI

From Headless to GUI

During training, we used GAMA in headless mode for speed. Now that the model is trained, we switch to the GAMA GUI to watch the agent in action.

Key Difference

Mode	Port	Speed	Display	Use
Headless	1001	Fast (no rendering)	None	Training
GUI	1000	Slow (renders display)	Yes	Testing / Visualization

---

Starting GAMA GUI

1. Open GAMA normally (double-click the application)
2. The GUI server runs on port 1000 by default — no extra setup needed

---

The Test Script

Loading the Trained Model

from train_forager import PPOAgent

agent = PPOAgent(state_dim=13, action_dim=2)
agent.load("saved_models/ppo_forager.pth")

This loads the neural network weights saved during training.

Connecting to GUI

The only change from training: the port number.

env = gym.make(
    "gama_gymnasium_env/GamaEnv-v0",
    gaml_experiment_path="path/to/forager_gym.gaml",
    gaml_experiment_name="gym_env",
    gama_ip_address="localhost",
    gama_port=1000,
)

Deterministic Evaluation

action, _, _ = agent.select_action(obs, test=True)

During training, the policy samples actions from a Normal distribution (adding noise for exploration). During testing, we set test=True to use the mean action — no randomness, just the best action the network has learned.

Slow Motion

time.sleep(0.1)  # 100ms delay between steps

We add a small delay between steps so the movement is visible in the GAMA display.

---

Running the Test

cd models/gym
python test_forager.py

Watch the GAMA display — the blue forager should navigate smoothly around obstacles to reach the green food!

---

Expected Console Output

==================================================
  Smart Forager - PPO Test (GUI)
==================================================
  Step 0: obs[:5]=[0.05, 0.05, 0.95, 0.52, 0.48] action=[0.73, 0.71]
  Step 1: obs[:5]=[0.07, 0.07, 0.93, 0.54, 0.50] action=[0.81, 0.78]
  ...

  Result: FOUND FOOD! | Steps: 49 | Reward: 92.6

Done.

---

Summary

Concept	Implementation
Continuous world	No grid, free {x, y} movement in 100×100 space
GAMA↔Python bridge	GymAgent species + gama-gymnasium WebSocket
Neural network policy	Custom ActorCritic with PyTorch Normal distribution
Headless training	gama-headless.bat -socket 1001 → fast, no GUI
GUI testing	Load .pth + connect to GAMA GUI (port 1000)
Action space	Box([-1,-1], [1,1]) — continuous velocity (dx, dy)
Observation space	Box([0]×13, [1]×13) — position + food direction + 8 sensors
Reward shaping	Distance delta bonus + food reward + living penalty

Key GAML Concepts Used

global, species, reflex, action, aspect, experiment, parameter, geometry, rectangle, circle, line, distance_to, towards, intersects, inter, cos, sin, list, map, loop, draw

Key Python Concepts Used

gymnasium, gym.make(), env.reset(), env.step(), torch.nn.Module, Normal distribution, ActorCritic, PPOAgent, asyncio

Key Files

File	Description
models/gym/forager_gym.gaml	GAMA model with GymAgent bridge
models/gym/train_forager.py	PPO training script (headless)
models/gym/test_forager.py	Testing script (GUI visualization)