Skip to content

Commit 3f6a336

Browse files
committed
docs: Update README.md with Physics simulations, CAD importer, WebSockets classroom, and Render deployment
1 parent 0552416 commit 3f6a336

1 file changed

Lines changed: 83 additions & 72 deletions

File tree

README.md

Lines changed: 83 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -1,109 +1,120 @@
1-
# Mindscape
1+
# Mindscape AI — AI Spatial Reasoning & STEM 3D Tutor
22

3-
Mindscape turns a static maths or physics worksheet into an interactive 3D lesson. Learners upload a diagram or type a question, inspect and manipulate the generated scene, then explain their reasoning to receive focused feedback rather than just an answer.
3+
Mindscape turns static STEM worksheets, 3D CAD models, and physics problems into an interactive, real-time 3D playground. Learners upload diagrams or CAD files, explore electric field vector mechanics, manipulate generated scenes, and participate in multi-user real-time 3D classrooms with live AI spatial tutoring.
44

5-
The flagship flow is a surface-area lesson: worksheet to labelled 3D solid, then exploration, prediction, explanation, and a follow-up challenge.
5+
---
66

7-
## Try the judge-safe demo
7+
## Key Capabilities
88

9-
Start the app and open [http://localhost:3000/?demo=true](http://localhost:3000/?demo=true). The guided cuboid surface-area lesson runs through the local planner and does not require API keys. It is designed for a reliable demo recording or judge review.
9+
### ⚡ 1. Advanced Interactive Physics & Vector Simulations
10+
- **Coulomb Force Vector Arrows**: Real-time 3D visualization of pairwise electrostatic forces ($F_{ij} = k \frac{q_i q_j}{r_{ij}^2} \hat{r}_{ij}$) with attraction/repulsion color dynamics and magnitude vectors.
11+
- **Dynamic Field Line Particle Flow**: Interactive RK4 streamline particle flow visualizing electric field lines.
12+
- **Physics Playground Simulation**: Play/Pause trajectory integration accelerating charges under electrostatic forces.
1013

11-
## Local setup
14+
### 📦 2. Custom 3D Model & CAD Importer (.gltf / .glb / .obj / .stl)
15+
- **Drag & Drop Viewport Uploading**: Learners and engineers can drag custom 3D files directly into the Three.js viewport.
16+
- **AI Spatial Analysis Metrics**: Automated geometry extraction (bounding box dimensions $dx, dy, dz$, surface area, estimated volume, vertex & face counts) fed directly to the AI Tutor for spatial reasoning.
1217

13-
### Requirements
18+
### 👥 3. Multi-User Real-Time 3D Classroom (WebSockets)
19+
- **Shared 3D Canvas Rooms**: Join via 6-character room code to connect teachers and learners live.
20+
- **Real-Time Spatial Sync**: Object creation, scaling, rotation, deletion, presence cursor indicators, and tutor hints sync instantly across devices via WebSockets (`wss://`).
21+
22+
---
1423

15-
- Node.js 20+
16-
- npm
17-
- Webcam for hand tracking
18-
- Microphone for push-to-talk voice mode
24+
## Quick Start & Local Setup
25+
26+
### Requirements
27+
- **Node.js 22+**
28+
- **npm**
29+
- Webcam (optional, for hand tracking)
30+
- Microphone (optional, for push-to-talk voice mode)
1931

20-
### Install
32+
### Installation
2133

2234
```bash
35+
# Install dependencies
2336
npm install
2437
pip install -r requirements.txt
2538
```
2639

27-
### Environment
40+
### Environment Configuration
2841

29-
Create `.env.local` in the project root:
42+
Create a `.env.local` file in the project root:
3043

3144
```env
3245
GEMINI_API_KEY=your_google_ai_studio_key
3346
GROQ_API_KEY=your_groq_api_key
3447
```
3548

36-
The app requires these keys for vision (Gemini), conversational tutoring (Groq), and transcription (Groq).
37-
38-
## Run
49+
### Development Server
3950

4051
```bash
4152
npm run dev
4253
```
4354

44-
Then open `http://localhost:3000`. Use `?demo=true` for the no-key guided worksheet demo.
55+
Open [http://localhost:3000](http://localhost:3000) in your browser. Use `?demo=true` for the offline guided worksheet demo.
4556

46-
## Verify
57+
### Quality & Test Suite
4758

4859
```bash
4960
npm run quality
5061
```
5162

52-
This runs linting and the complete automated test suite.
53-
54-
## Built with Codex and GPT-5.6
55-
56-
Codex and GPT-5.6 were used to stabilize the interactive lesson flow, repair the streaming voice lifecycle, add a no-key demo path, strengthen test coverage, and refine the submission-ready product experience. The project keeps deterministic lesson and scene fallbacks so the core learning journey remains demonstrable when external model services are unavailable.
57-
58-
## Architecture
59-
60-
### High-level system design
61-
62-
```text
63-
+---------------------------------------------------------------+
64-
| Frontend |
65-
| Vanilla JS + Three.js |
66-
| |
67-
| - Question input (text, image, screenshot) |
68-
| - 3D scene rendering |
69-
| - Tutor panel |
70-
| - Voice UI (Browser Web Speech API) |
71-
| - Hand tracking |
72-
| - KaTeX rendering |
73-
| - Real-time "Limited Mode" alerts |
74-
+---------------------------------------------------------------+
63+
Runs ESLint and the complete automated unit test suite (170+ tests).
64+
65+
---
66+
67+
## ☁️ Free Cloud Deployment (Render)
68+
69+
Mindscape AI includes a production multi-stage `Dockerfile` (Node 22) and `render.yaml` blueprint for 1-click free deployment on **Render** with full WebSocket (`wss://`) support.
70+
71+
### 1-Click Render Deployment
72+
1. Push code to your GitHub repository.
73+
2. Log into [Render Dashboard](https://dashboard.render.com/) $\rightarrow$ Click **New +** $\rightarrow$ **Blueprint**.
74+
3. Select your repository. Render will automatically detect `render.yaml`, build Vite production assets, and launch the Node.js + WebSocket Web Service for free.
75+
76+
---
77+
78+
## System Architecture
79+
80+
```text
81+
+---------------------------------------------------------------+
82+
| Frontend |
83+
| Vanilla JS + Three.js + WebSockets |
84+
| |
85+
| - Question & CAD Input (text, image, .gltf, .obj, .stl) |
86+
| - Physics Simulation & Coulomb Vector Arrow Overlay |
87+
| - Multi-User Real-Time 3D Classroom UI |
88+
| - AI Tutor & KaTeX math rendering |
89+
| - Hand tracking & 3D gesture controls |
90+
+---------------------------------------------------------------+
7591
|
76-
| HTTP / SSE
92+
| HTTP / SSE / WebSockets (ws://)
7793
v
78-
+---------------------------------------------------------------+
79-
| Backend |
80-
| Node.js + Hono API |
81-
| |
82-
| - Request validation |
83-
| - Gemini & Groq Hybrid integration |
84-
| - 3-Tier Model Failover (Gemini -> Groq 70B -> Groq 8B) |
85-
| - Text-as-Vision Fallback (Rescues scene interpretation) |
86-
| - SceneSpec generation (Strict JSON enforcement) |
87-
| - Tutor streaming |
88-
| - Voice pipeline coordination (Whisper STT) |
89-
+---------------------------------------------------------------+
94+
+---------------------------------------------------------------+
95+
| Backend Server |
96+
| Node.js 22 + Hono API + WebSocket Room Manager |
97+
| |
98+
| - Real-time Classroom Room Broadcasting |
99+
| - Gemini & Groq Hybrid AI Orchestration |
100+
| - 3D CAD Geometric Metrics Extraction |
101+
| - 3-Tier Model Failover (Gemini -> Groq 70B -> Groq 8B) |
102+
| - SceneSpec generation & Tutor Streaming |
103+
+---------------------------------------------------------------+
90104
|
91105
+--------------+--------------+
92106
v v
93-
+-----------------------+ +-----------------------+
94-
| Google Gemini | | Groq (Llama 3) |
95-
| | | |
96-
| - Primary Vision | | - Primary Reasoning |
97-
| - Primary Planning | | - Whisper STT (v3) |
98-
| - Failover Chat | | - 70B Scene Planning |
99-
| | | - Visual Failover |
100-
+-----------------------+ +-----------------------+
101-
```
102-
103-
## Hardened Failover System
104-
105-
The system is built for hackathon-grade resiliency:
106-
- **Vision Fallback**: If Gemini 2.0 Flash hits quota or fails, the system automatically falls back to **Llama 3.2 11B Vision** on Groq.
107-
- **Interpretation Fallback**: If all vision models fail to interpret a diagram, the system uses the high-capacity **Llama 3.3 70B** to interpret the question text alone, ensuring a valid 3D scene is still built.
108-
- **Search Fallback**: Semantic search (Gemini Embeddings) automatically falls back to **Lexical Keyword Search** if the API key is restricted or out of quota.
109-
- **Transparency**: The UI displays a "Limited Mode" warning in the evidence panel whenever the system is operating in a fallback state.
107+
+-----------------------+ +-----------------------+
108+
| Google Gemini | | Groq (Llama 3) |
109+
| | | |
110+
| - Primary Vision | | - Primary Reasoning |
111+
| - Primary Planning | | - Whisper STT |
112+
| - Failover Chat | | - 70B Scene Planning |
113+
+-----------------------+ +-----------------------+
114+
```
115+
116+
---
117+
118+
## License & Credits
119+
120+
Built with Three.js, Hono, Node.js 22, Google Gemini API, and Groq SDK.

0 commit comments

Comments
 (0)