Three ML-Agents

A JavaScript / Python re-implementation of the core Unity ML-Agents examples. Test out the Live Demos.

The goal of this repository is to make the ideas from Unity ML-Agents easily approachable from the browser (React + Three.js) and from lightweight Python back-ends (FastAPI) to be used for future study.

Examples

View all the demos on the live demos page: Live Demos.

Select demos are here.

MineCraft

Live demo

Glider

Live demo

Astrodynamics

Live demo

Fish

Live demo

Self-driving Car (Interpretability)

Live demo

Labyrinth (NetHack)

Live demo

Demo	Live Demo
Basic 1-D Move to Goal	Live Demo
3DBall Balance	Live Demo
GridWorld Navigation	Live Demo
Push-Block	Live Demo
Wall Jump	Live Demo
Ant (Crawler)	Live Demo
Swimmer / Worm	Live Demo
Brick Break	Live Demo
Food Collector	Live Demo
Bicycle	Live Demo
Intersection	Live Demo

Three ML-Agents

The library in Python+threejs should migrate easily matching from the previous ML-Agents if you're familiar with those examples.

Feature	Status	Notes
Vector Observations	✅ Stable	Standard numeric observations supported.
Visual Observations	✅ Beta	CameraSensor implemented. Warning: Uses Base64/JSON (Prototype efficiency).
Ray Perception	✅ Stable	RayPerceptionSensor simulates Lidar/Raycasts including tag detection.
Side Channels	✅ Stable	Support for EngineConfiguration (timescale), Stats (logging), and EnvironmentParameters.
Decision Requester	✅ Stable	Agents can request decisions at customized intervals (skip-frames).

Architecture Note: The migration uses bridge from Python to Javascript:

• Python: api/mlagents_bridge (Standalone env compatible with mlagents-envs).
• JavaScript: client/src/libs/ml-agents (Modular Agent/Academy/Sensor architecture).

---

Project layout

three-mlagents/
│
├─ client/         ← Vite + React + Three.js front-end (examples live here)
│   ├─ src/examples
│   │   ├─ Index.jsx  ← landing page listing all examples
│   │   └─ Basic.jsx  ← 1-D "move-to-goal" environment matching Unity Basic
│   └─ src/libs/ml-agents  ← New JS Client SDK (Academy, Agent, Sensors)
│
├─ api/            ← FastAPI micro-service (Python ≥3.9)
│   ├─ mlagents_bridge/    ← New Python Environment Bridge
│   ├─ main.py      ← gym-style REST API for each environment
│   └─ requirements.txt

You only need client and api to run the demos. The ml-agents directory remains untouched so you can cross-reference the original C#/Unity code (see Examples/Basic/Scripts/BasicController.cs for this particular demo).

---

Quick start

1. Front-end (browser)

cd client
npm install        # installs React, Three.js, @react-three/fiber, @react-three/drei …
npm run dev        # opens http://localhost:5173

You will land on / – a list of examples. Click /basic to open the first scene. Use ← / → to move the agent cube; rewards replicate the logic of the original BasicController script:

• Each step: −0.01
• Small goal (position 7): +0.1 and episode terminates
• Large goal (position 17): +1 and episode terminates

2. Back-end (optional)

For browser-only play the API is not required. If you want to let external RL algorithms interact with the environment, spin up the FastAPI service:

cd api
pip install -r requirements.txt
uvicorn main:app --reload

Endpoints:

• POST /basic/reset → { "position": 10 }
• POST /basic/step → { "position": int, "reward": float, "done": bool }

The contract mirrors OpenAI Gym (position plays the role of the observation).

---

MineCraft RL + LLM Example - Additional Setup

The MineCraft example (#11) combines reinforcement learning with large language models (LLMs) for intelligent agent behavior. This requires additional setup beyond the basic installation.

Option 1: Local Ollama (Recommended for Development)

1. Install Ollama: Download and install from ollama.ai

2. Pull the required model:

   ollama pull gemma3n:latest

3. Configure the example: In api/examples/minecraft.py, set:

   USE_LOCAL_OLLAMA = True

4. Start Ollama: Make sure Ollama is running locally (usually starts automatically)

Option 2: OpenRouter (Cloud-based)

1. Get an OpenRouter API key: Sign up at openrouter.ai and get your API key

2. Set environment variable:

   export OPENROUTER_API_KEY="your_api_key_here"

3. Configure the example: In api/examples/minecraft.py, set:

   USE_LOCAL_OLLAMA = False

4. Note: This option uses anthropic/claude-sonnet-4 and requires credits/tokens

Additional Dependencies

The MineCraft example also requires additional Python packages for embeddings and LLM functionality. Make sure these are included in your api/requirements.txt:

sentence-transformers  # for text embeddings
openai                 # for LLM API calls (if using OpenRouter)

Running the MineCraft Example

1. Start the API with LLM support:

   cd api
   pip install -r requirements.txt
   uvicorn main:app --reload

2. Open the frontend and navigate to the MineCraft example

3. The agents will use LLMs for strategic decision-making, communication, and trading while using RL for basic movement and mining actions.

---

---

License

MIT © 2025