Skip to content

Commit d1f834a

Browse files
stephengpopestephengpope
committed
adding claude.md
1 parent c70c646 commit d1f834a

1 file changed

Lines changed: 248 additions & 0 deletions

File tree

‎CLAUDE.md‎

Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Overview
6+
7+
No-Code Architects Toolkit API is a Flask-based media processing API that handles audio/video conversion, transcription, translation, captioning, and cloud storage integration. It supports deployment on Docker, Google Cloud Platform, and Digital Ocean.
8+
9+
## Architecture
10+
11+
### Core Components
12+
13+
- **[app.py](app.py)** - Main Flask application with queue-based task processing
14+
- Creates task queue for async job processing
15+
- Provides `queue_task` decorator for route handlers
16+
- Supports GCP Cloud Run Jobs for long-running tasks
17+
- Auto-registers blueprints from `routes/` directory
18+
19+
- **[app_utils.py](app_utils.py)** - Core utilities
20+
- `validate_payload()` - JSON schema validation decorator
21+
- `queue_task_wrapper()` - Wraps routes for queue processing
22+
- `discover_and_register_blueprints()` - Auto-discovers and registers Flask blueprints
23+
- `log_job_status()` - Logs job status to LOCAL_STORAGE_PATH/jobs
24+
25+
- **[config.py](config.py)** - Environment configuration
26+
- Validates required environment variables per storage provider
27+
- Configures API_KEY, storage paths, and cloud credentials
28+
29+
### Request Flow
30+
31+
1. Request hits route in `routes/v1/{category}/{action}.py`
32+
2. `@authenticate` decorator validates X-API-Key header
33+
3. `@validate_payload()` validates JSON against schema
34+
4. `@queue_task_wrapper()` determines processing path:
35+
- **No webhook_url**: Execute synchronously, return immediately
36+
- **With webhook_url**: Queue task, return 202, send webhook when done
37+
- **GCP_JOB_NAME set + webhook_url**: Trigger Cloud Run Job, return 202
38+
- **CLOUD_RUN_JOB env set**: Execute synchronously in job context
39+
40+
5. Route calls service function in `services/v1/{category}/{action}.py`
41+
6. Service processes media, uploads to cloud storage, returns result
42+
7. Route returns tuple: `(response_data, endpoint_string, status_code)`
43+
44+
### Job Processing Modes
45+
46+
**In-Process Queue** (Default)
47+
- Single queue per worker process
48+
- Background thread processes tasks sequentially
49+
- MAX_QUEUE_LENGTH env var limits queue size (0 = unlimited)
50+
51+
**GCP Cloud Run Jobs** (Optional)
52+
- Set GCP_JOB_NAME and GCP_JOB_LOCATION to enable
53+
- Requires webhook_url in request payload
54+
- Triggers Cloud Run Job with endpoint and payload as env vars
55+
- Job executes task independently and sends webhook
56+
57+
**Synchronous** (No Queue)
58+
- Used when webhook_url not provided
59+
- Request blocks until processing completes
60+
61+
### Dynamic Route Registration
62+
63+
Routes are auto-discovered from `routes/` directory. No manual registration needed in [app.py](app.py).
64+
65+
**Blueprint Convention:**
66+
```python
67+
from flask import Blueprint
68+
from app_utils import validate_payload, queue_task_wrapper
69+
from services.authentication import authenticate
70+
71+
v1_category_action_bp = Blueprint('v1_category_action', __name__)
72+
73+
@v1_category_action_bp.route('/v1/category/action', methods=['POST'])
74+
@authenticate
75+
@validate_payload(schema_dict)
76+
@queue_task_wrapper(bypass_queue=False)
77+
def action_handler(job_id, data):
78+
"""
79+
Args:
80+
job_id (str): Unique job identifier
81+
data (dict): Request JSON payload
82+
83+
Returns:
84+
Tuple[dict, str, int]: (response_data, endpoint_path, status_code)
85+
"""
86+
# Implementation
87+
return result, "/v1/category/action", 200
88+
```
89+
90+
See [docs/adding_routes.md](docs/adding_routes.md) for detailed guide.
91+
92+
### Cloud Storage Abstraction
93+
94+
[services/cloud_storage.py](services/cloud_storage.py) provides unified interface:
95+
- Detects provider from environment variables
96+
- GCP: Requires GCP_SA_CREDENTIALS, GCP_BUCKET_NAME
97+
- S3: Requires S3_ENDPOINT_URL, S3_ACCESS_KEY, S3_SECRET_KEY, S3_BUCKET_NAME, S3_REGION
98+
- Digital Ocean: Extracts bucket/region from endpoint URL if not provided
99+
100+
Service functions call `upload_file(local_path)` which returns public URL.
101+
102+
## Development Commands
103+
104+
### Local Development
105+
106+
```bash
107+
# Install dependencies
108+
pip install -r requirements.txt
109+
110+
# Run development server (default port 8080)
111+
python app.py
112+
113+
# Or use gunicorn
114+
gunicorn --config gunicorn.conf.py app:app
115+
```
116+
117+
### Docker
118+
119+
```bash
120+
# Build image
121+
docker build -t no-code-architects-toolkit .
122+
123+
# Run container
124+
docker run -p 8080:8080 \
125+
-e API_KEY=your_key \
126+
-e S3_ENDPOINT_URL=... \
127+
-e S3_ACCESS_KEY=... \
128+
-e S3_SECRET_KEY=... \
129+
-e S3_BUCKET_NAME=... \
130+
-e S3_REGION=... \
131+
no-code-architects-toolkit
132+
133+
# Or use docker-compose
134+
docker-compose up
135+
```
136+
137+
### Testing
138+
139+
The API uses Postman for testing. Template available at: https://bit.ly/49Gkh61
140+
141+
Test endpoint:
142+
```bash
143+
curl -X POST http://localhost:8080/v1/toolkit/test \
144+
-H "X-API-Key: your_key"
145+
```
146+
147+
## Environment Variables
148+
149+
**Required:**
150+
- `API_KEY` - Authentication key for all endpoints
151+
152+
**Storage (choose one):**
153+
154+
GCP Storage:
155+
- `GCP_SA_CREDENTIALS` - Service account JSON credentials
156+
- `GCP_BUCKET_NAME` - GCS bucket name
157+
158+
S3-Compatible:
159+
- `S3_ENDPOINT_URL` - S3 endpoint URL
160+
- `S3_ACCESS_KEY` - Access key
161+
- `S3_SECRET_KEY` - Secret key
162+
- `S3_BUCKET_NAME` - Bucket name
163+
- `S3_REGION` - Region (or "None" for some providers)
164+
165+
**Optional:**
166+
- `LOCAL_STORAGE_PATH` - Temp file storage (default: /tmp)
167+
- `MAX_QUEUE_LENGTH` - Max concurrent tasks (default: 0/unlimited)
168+
- `GUNICORN_WORKERS` - Worker processes (default: CPU cores + 1)
169+
- `GUNICORN_TIMEOUT` - Worker timeout seconds (default: 30)
170+
- `GCP_JOB_NAME` - Cloud Run Job name for offloading
171+
- `GCP_JOB_LOCATION` - Cloud Run Job region (default: us-central1)
172+
173+
## Key Patterns
174+
175+
### Route Structure
176+
- Routes in `routes/v1/{category}/{action}.py`
177+
- Services in `services/v1/{category}/{action}.py`
178+
- Documentation in `docs/{category}/{action}.md`
179+
180+
### Return Format
181+
All routes return: `(response_dict, endpoint_string, http_status_code)`
182+
183+
The `queue_task` decorator wraps this into full response with metadata:
184+
```json
185+
{
186+
"code": 200,
187+
"id": "user_provided_id",
188+
"job_id": "uuid",
189+
"response": {...},
190+
"message": "success",
191+
"run_time": 1.234,
192+
"queue_time": 0.567,
193+
"total_time": 1.801,
194+
"pid": 12345,
195+
"queue_id": 140123456789,
196+
"queue_length": 3,
197+
"build_number": "204"
198+
}
199+
```
200+
201+
### Job Status Tracking
202+
Job status files written to `{LOCAL_STORAGE_PATH}/jobs/{job_id}.json`
203+
204+
Status values: `queued`, `running`, `done`, `failed`, `submitted`
205+
206+
### Webhook Pattern
207+
When `webhook_url` provided in request:
208+
- Task queued, immediate 202 response
209+
- On completion, POST result to webhook_url
210+
- Used to avoid timeouts on long-running tasks
211+
212+
### FFmpeg Usage
213+
Most media processing uses FFmpeg via [ffmpeg-python](https://pypi.org/project/ffmpeg-python/) library.
214+
215+
Example from services:
216+
```python
217+
import ffmpeg
218+
219+
input_stream = ffmpeg.input(video_path)
220+
output = ffmpeg.output(input_stream, output_path, **options)
221+
ffmpeg.run(output, overwrite_output=True)
222+
```
223+
224+
## Adding New Features
225+
226+
1. Create service function in `services/v1/{category}/{action}.py`
227+
2. Create route in `routes/v1/{category}/{action}.py` following blueprint pattern
228+
3. Add JSON schema validation to route
229+
4. Service should download inputs, process, upload to cloud storage
230+
5. Return `(result_dict, endpoint_string, status_code)` from route
231+
6. Add documentation to `docs/{category}/{action}.md`
232+
7. Update [README.md](README.md) with new endpoint
233+
234+
See [docs/adding_routes.md](docs/adding_routes.md) for full guide.
235+
236+
## Contributing
237+
238+
- Submit PRs to `build` branch (not main)
239+
- Use auto-versioning: builds auto-increment via GitHub Actions
240+
- Update README.md when adding new endpoints
241+
- Follow GPL-2.0 license (see [LICENSE](LICENSE))
242+
243+
## Deployment Guides
244+
245+
- Digital Ocean App Platform: [docs/cloud-installation/do.md](docs/cloud-installation/do.md)
246+
- Google Cloud Run: [docs/cloud-installation/gcp.md](docs/cloud-installation/gcp.md)
247+
- General Docker: [docker-compose.md](docker-compose.md)
248+
- Local with MinIO + n8n: [docker-compose.local.minio.n8n.md](docker-compose.local.minio.n8n.md)

0 commit comments

Comments
 (0)