Skip to content

Commit 3c0cd2d

Browse files
authored
Merge pull request #39 from viruslox/update-docs-architecture-config-templates-3251558865566657047
docs: update architecture, readme, and config templates to reflect refactoring
2 parents ae07731 + 081b4e2 commit 3c0cd2d

7 files changed

Lines changed: 98 additions & 8 deletions

ARCHITECTURE.md

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,21 @@ The project has been refactored from a monolithic bash structure into a compiled
3333

3434
1. **`VLX_FrameFlow` (The Client):** Exclusively deployed on Single Board Computers (like Orange Pi 5 Plus or Radxa Rock 5T). Responsible for hardware interactions: capturing video via v4l2, managing hostapd/networkd, and gathering GPS data. It serves the mTLS-protected API.
3535
2. **`VLX_FrameFlow_SRV` (The Server):** Exclusively deployed on a Virtual Private Server (VPS). Lightweight, focusing strictly on relaying traffic, receiving bonded connections, and enforcing UFW firewall rules.
36-
3. **`vlx_frontend` (The UI):** A standalone web server encapsulating a pre-built Svelte SPA (`//go:embed`). Designed to run remotely on an operator's machine or in the cloud to manage the Field Unit via secure APIs.
36+
3. **`vlx_frontend` (The UI):** A standalone web server encapsulating a pre-built Svelte SPA (`//go:embed`). Designed to run remotely on an operator's machine or in the cloud to manage the Field Unit via secure APIs. The frontend utilizes parallel REST polling (`Promise.all`) to dynamically fetch backend state, features a semantic parser for translating systemd string statuses (e.g., 'active', 'inactive', 'failed'), organizes UI components via CSS Grid layout, and streams text console output utilizing an `ansi-to-html` converter for colorized logs.
37+
38+
## API Framework & Routing
39+
40+
The core API has transitioned to utilizing the highly performant `Gin` framework (`github.com/gin-gonic/gin`). HTTP routes are registered using native Gin handler signatures (`func(c *gin.Context)`) rather than wrapping standard `http.HandlerFunc` components. Global CORS middleware handles OPTIONS requests automatically.
41+
42+
### Operational Endpoints
43+
44+
The unprivileged user interacts with the backend components via a standardized REST API structured primarily as `/api/v1/<module>/{start,stop,status,reset}`. Key operational endpoints include:
45+
46+
- **`/api/v1/client/reset`**: Initiates a full reset of client networking and bonding.
47+
- **`/api/v1/gps/start` \| `stop` \| `status`**: Manages the transient systemd user unit for the GPS telemetry process.
48+
- **`/api/v1/mediamtx/start` \| `stop` \| `status`**: Controls the local MediaMTX static user service.
49+
- **`/api/v1/cameraman/start` \| `stop` \| `status`**: Orchestrates FFmpeg encoding pipelines directly to the MediaMTX API.
50+
- **`/api/v1/ap/start` \| `stop` \| `status`**: Triggers internal privilege escalation to manipulate `hostapd` and network interfaces.
3751

3852
## Unified Configuration Paradigm
3953

@@ -65,11 +79,23 @@ To secure the connection between the remote `vlx_frontend` and the SBC's `VLX_Fr
6579

6680
### "Build as User, Run as Root"
6781

68-
To maintain security and FHS compliance, the suite enforces a strict compilation and deployment workflow:
82+
The suite enforces a strict dichotomy between Client and Server roles regarding execution privilege and systemd architecture:
83+
6984
1. **Build:** Unprivileged compilation via `build.sh`.
7085
2. **Install:** Executing the compiled binary as root triggers `internal/sysutils.InstallBinary()`.
7186
3. **Deploy:** The binaries place themselves into `/opt/VLX_FrameFlow/bin/` and configure templates in `/opt/VLX_FrameFlow/etc/`.
72-
4. **Run:** Background services execute as the dedicated, unprivileged `$FRAMEFLOW_USER` via `systemd --user` units to limit the blast radius.
87+
4. **Run (CLIENT):** On the SBC/Field Unit, background services must strictly execute as the dedicated, unprivileged `$FRAMEFLOW_USER`. Systemd management uses User-Space Systemd (`systemctl --user`). Unit files are generated in `~/.config/systemd/user/` and explicitly target `default.target`. **Important:** Systemd Lingering (`loginctl enable-linger`) must be enabled by root during initial setup to allow background execution without an active SSH session.
88+
5. **Run (SERVER):** On the VPS/Relay Node, the main daemon strictly requires `root` execution (UID 0) to orchestrate routing and UFW firewall rules. Systemd management uses System-Space Systemd (`/etc/systemd/system/`) and targets `multi-user.target`. To maintain security, the server relies on **Privilege Dropping** (`User=`, `Group=`) inside the unit templates for specific exposed services (like MediaMTX) to minimize the blast radius.
89+
90+
### AP Module Privilege Escalation Pattern
91+
92+
The Access Point (AP) module within the Client architecture necessitates modifying system-level network configurations (`hostapd`, `systemd-networkd`, `systemd-resolved`) which requires root access. However, the Client CLI operates as an unprivileged user.
93+
94+
To bridge this gap securely, the suite implements a strict internal privilege escalation pattern:
95+
- Unprivileged CLI commands wrap themselves in `sudo` to call hidden internal operations (e.g., `_ap_system_ops`).
96+
- These hidden operations are protected by absolute root guards (`if os.Geteuid() != 0 { log.Fatal(...) }`).
97+
- Once inside the root context, functions executing system commands (like restarting services via `sysutils.RunCommand`) do not redundantly include `sudo`.
98+
7399

74100

75101
### Server API Command Forwarding
@@ -88,6 +114,23 @@ The dual-protocol nature ensures network reliability without cross-contamination
88114
- **Shadowsocks (via MPTCP):** Handles all bonded outbound TCP internet traffic. This is used by the Client to reach the broader internet transparently via the proxy.
89115
- **MLVPN (UDP):** Handles the high-bandwidth SRT streams and the secure, bidirectional internal telemetry and API command routing between the Server (`10.1.10.1`) and Client (`10.1.10.2`).
90116

117+
## Module-Specific Behaviors
118+
119+
The system implements several highly specific optimizations and fail-safes per module:
120+
121+
### Cameraman
122+
- **Strict URL Parsing:** Incorporates robust `net/url` parsing logic to validate SRT stream IDs.
123+
- **Fallback Format Selection:** Employs a linear absolute-difference algorithm against user-defined maximums (`CAM_MAX_RESOLUTION`, `CAM_MAX_FPS`) to intelligently select fallback video formats (preferring H.264 copy, then MJPEG, then YUYV) when exact formats are unavailable.
124+
- **SQLite Concurrency:** The backend SQLite database (`modernc.org/sqlite`) is configured with `_pragma=journal_mode(WAL)` and `_pragma=busy_timeout(5000)` in its DSN to inherently prevent 'database is locked' boot race conditions during concurrent module initialization.
125+
126+
### GPS Telemetry
127+
- **Socket Draining:** Implements a non-blocking TCP socket drain pattern against `gpsd` to prevent TCP buffer desynchronization and data lag typical of high-frequency positioning data.
128+
- **Rate Limiting:** Enforces a strict 5-second rate limit on HTTP POST transmissions to external endpoints.
129+
- **Transient Cleanup:** Actively manages transient systemd units, executing `systemctl --user reset-failed` during cleanup to prevent unit name conflicts.
130+
131+
### MediaMTX & Bonding (v2ray)
132+
- **Dynamic Architecture Resolution:** During initialization and updates, the system dynamically checks `runtime.GOARCH` to resolve and download correct asset architectures. This strictly prevents "Exec format error" failures when deploying across heterogeneous ARM and x86_64 fleets.
133+
91134
## Filesystem Structure
92135

93136
Following Linux File Hierarchy Standard (FHS) best practices, the global installation path is centralized:

README.md

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,13 @@ VLX FrameFlow is a modular Go-based suite designed to transform Debian-based **S
1616

1717
The suite operates via a main installation entry point compiled into a Go binary, internal Go packages, and runtime service managers. Process control is handled via **`systemd`**.
1818

19+
### Execution Privilege & Systemd Architecture
20+
21+
The system enforces a strict operational dichotomy based on the deployed role:
22+
23+
- **CLIENT (SBC/Field Unit):** Strictly operates as an unprivileged dedicated user (`frameflow_user`). Systemd management utilizes User-Space Systemd (`systemctl --user`), stores unit files in `~/.config/systemd/user/`, and explicitly targets `default.target`. Systemd Lingering must be enabled by root during installation.
24+
- **SERVER (VPS/Relay Node):** Strictly requires `root` execution (UID 0) to manage routing and firewall rules. Systemd management utilizes System-Space Systemd (`/etc/systemd/system/`) and targets `multi-user.target`. It relies on Privilege Dropping (`User=`, `Group=`) inside the unit templates to secure exposed services like MediaMTX.
25+
1926
The suite uses a **multi-protocol bonding architecture**: UDP traffic is routed exclusively via MLVPN, and TCP traffic is aggregated using MPTCP with Shadowsocks-libev and v2ray-plugin acting as a transparent proxy.
2027

2128
### Core Structure
@@ -26,10 +33,15 @@ The project has evolved into a multi-binary ecosystem to ensure role-specific fu
2633
| :--- | :--- |
2734
| **`VLX_FrameFlow`** | Client binary exclusively for SBC tasks (FFmpeg, AP, Storage, API). |
2835
| **`VLX_FrameFlow_SRV`** | Server binary exclusively for VPS tasks (Relay node, Firewall). |
29-
| **`vlx_frontend`** | Remote-capable standalone UI server. |
36+
| **`vlx_frontend`** | Remote-capable standalone UI server (Svelte SPA). |
3037
| **`config/`** | Configuration templates and maintenance scripts. |
3138
| **`internal/`** | Core logic (storage, system, network, package management). |
3239

40+
### API & Frontend
41+
The backend API is built using the highly performant **Gin** framework (`github.com/gin-gonic/gin`) serving a standardized REST structure (`/api/v1/...`).
42+
43+
The **Control Panel Frontend** is a compiled **Svelte** application. It dynamically polls the backend via parallel REST calls (`Promise.all`), features a semantic parser translating raw systemd statuses into visual states, and provides real-time streaming text consoles utilizing `ansi-to-html`.
44+
3345
---
3446

3547
## Installation
@@ -132,7 +144,7 @@ Available commands:
132144
- **`server start` / `server status` / `server stop`**: Manage server components.
133145
- **`server api start` / `server api status` / `server api stop`**: Manage the local API relay for forwarding commands to the remote Client via MLVPN.
134146
- **`client start` / `client status` / `client stop`**: Manage client components.
135-
- **`client reset`**: Restarts networking and bonding services.
147+
- **`client reset`**: Restarts networking and bonding services. (Also exposed via `/api/v1/client/reset`).
136148
- **`bonding`**: Displays MPTCP proxy and MLVPN tunnel status.
137149
- **`AP start`**: Activates HostAP (hotspot) on the first Wi-Fi interface.
138150
- **`AP stop`**: Stops HostAP and switches the first Wi-Fi interface back to managed client mode.
@@ -159,7 +171,7 @@ Manages video encoding pipelines. This service is best utilized in combination w
159171
./vlx_frameflow mediamtx <start|stop|status>
160172
```
161173

162-
Manages the local **MediaMTX** server (SRT protocol only). Combining this service with the suite's Access Point (AP) mode is crucial for achieving optimal performance and reliability when interfacing with devices running rigid or proprietary software, such as GoPro cameras.
174+
Manages the local **MediaMTX** server (SRT protocol only). Combining this service with the suite's Access Point (AP) mode is crucial for achieving optimal performance and reliability when interfacing with devices running rigid or proprietary software, such as GoPro cameras. Also exposed via REST (`/api/v1/mediamtx/start|stop|status`).
163175

164176
**Functions:**
165177

@@ -172,7 +184,7 @@ Manages the local **MediaMTX** server (SRT protocol only). Combining this servic
172184
./vlx_frameflow gps <start|stop|status>
173185
```
174186

175-
Manages GPS and telemetry services. This module acts as a resilient data pipeline, ensuring real-time location and telemetry data are consistently processed and transmitted regardless of network fluctuations.
187+
Manages GPS and telemetry services. This module acts as a resilient data pipeline, ensuring real-time location and telemetry data are consistently processed and transmitted regardless of network fluctuations. Also exposed via REST (`/api/v1/gps/start|stop|status`).
176188

177189
- Controls **gpsd**.
178190
- Auto-detects USB / serial GPS hardware.

config/frameflow.settings.template

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,37 @@
11
# VLX FrameFlow User Profile
2+
# The dedicated, unprivileged user that runs the background services.
23
FRAMEFLOW_USER="frameflow"
4+
5+
# Installation Paths
6+
# Paths dynamically resolved via os.Getenv("VLXsuite_DIR") in code, defaulting to:
37
VLXsuite_DIR="/opt/VLX_FrameFlow"
8+
# Note: Architecture suffixes (e.g. AMD64 vs ARM64) for MediaMTX are handled automatically
9+
# by the daemon's runtime.GOARCH resolution and do not need to be appended here.
410
MEDIAMTX_DIR="/opt/mediamtx"
511

612
# Wi-Fi AP Password
13+
# Automatically generated if left empty.
714
AP_PASSWORD=""
815

916
# GPS Tracker
1017
gps_target_url=""
1118

1219
# Network settings
20+
# Specifies role-aware privilege scoping. Do not manually force unless required.
1321
FRAMEFLOW_ROLE=""
1422
MLVPN_SERVER_IP=""
23+
# Supports native Dual-Stack TCP bonding (comma-separated IPv4, IPv6)
1524
SHADOWSOCKS_SERVER_IPS=""
1625
MPTCP_PROXY_PASS=""
1726
MLVPN_KEY=""
1827

1928
# Cameraman
29+
# DSN utilizes _pragma=journal_mode(WAL)&_pragma=busy_timeout(5000) internally to prevent boot race conditions.
2030
DB_DSN="/opt/VLX_FrameFlow/var/frameflow.db"
2131
# SRT target
32+
# Undergoes strict net/url validation before the encoding pipeline starts.
2233
SRT_URL="srt://frameflow-server.example:8890?streamid=publish:cam"
34+
# User-defined ceilings that the linear absolute-difference fallback algorithm strictly respects.
2335
CAM_MAX_RESOLUTION="1920x1080"
2436
CAM_MAX_FPS="30"
2537

config/frameflow_srv.settings.template

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,17 @@
11
# VLX FrameFlow Server Profile
2+
# The dedicated, unprivileged user that runs background services inside privilege-dropped systemd units.
23
FRAMEFLOW_USER="frameflow"
4+
5+
# Installation Paths
6+
# Paths dynamically resolved via os.Getenv("VLXsuite_DIR") in code, defaulting to:
37
VLXsuite_DIR="/opt/VLX_FrameFlow"
8+
# Note: Architecture suffixes (e.g. AMD64 vs ARM64) for MediaMTX are handled automatically
9+
# by the daemon's runtime.GOARCH resolution and do not need to be appended here.
410
MEDIAMTX_DIR="/opt/mediamtx"
511

612
# Network settings
13+
# Explicitly sets the system to require root execution for main orchestrator daemons,
14+
# while utilizing /etc/systemd/system/ and multi-user.target for unit files.
715
FRAMEFLOW_ROLE="SERVER"
816
MPTCP_PROXY_PASS=""
917
MLVPN_KEY=""

config/frontend.settings.template

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,21 @@
1+
# Standalone Web Server Binding Configuration
12
# (optional, if empty fallback to localhost)
23
bind_address=0.0.0.0
34
# (optional, if empty fallback to 8080)
45
bind_port=8080
6+
57
# Frontend GUI user credential
68
FF_GUI_USER=frontend
79
FF_GUI_PASS=frontend
10+
11+
# Target Backend Connection Parameters (Client API)
812
# (connection to backend: optional, if empty set to localhost)
913
backend_address=127.0.0.1
1014
# (connection to backend: optional, if empty set to 9090)
1115
backend_port=9090
16+
17+
# Zero-Trust mTLS Credentials
18+
# Used by the Svelte Control Panel to authenticate against the backend API.
1219
# Client certificate path for frontend (optional, required if backend enforces mTLS)
1320
client_crt=/opt/VLX_FrameFlow/certs/frontend01.crt
1421
# Client key path for frontend (optional, required if backend enforces mTLS)

config/mediamtx_client.settings.template

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
###############################################
22
# VLX FrameFlow - MediaMTX Configuration Template (Client)
3+
# Note: This node operates natively as a static user service using systemctl --user,
4+
# exclusively running as the dedicated unprivileged user via ~/.config/systemd/user/.
35
###############################################
46
logLevel: info
57
logDestinations: [syslog]
@@ -79,5 +81,7 @@ pathDefaults:
7981

8082
paths:
8183
~^wificam:
84+
# Acts as an RTMP-to-SRT auto-forwarder
8285
runOnReady: ffmpeg -i rtmp://localhost:$RTMP_PORT/$MTX_PATH -c copy -f srt "srt://10.1.10.1:8890?streamid=publish:$MTX_PATH"
8386
~^cameraman:
87+
# Dynamically configured via the API. Operates exclusively over SRT (V4L2/ALSA direct-to-SRT ingest) without WebRTC multiplexing.

config/mediamtx_server.settings.template

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
###############################################
2-
# VLX FrameFlow - MediaMTX Configuration Template (Client)
2+
# VLX FrameFlow - MediaMTX Configuration Template (Server)
3+
# Note: This node operates inside a System-Space Systemd unit (/etc/systemd/system/).
4+
# Privilege Dropping (User=, Group=) is enforced inside the generated unit to limit blast radius.
35
###############################################
46
logLevel: info
57
logDestinations: [syslog]
@@ -86,5 +88,7 @@ pathDefaults:
8688
runOnRecordSegmentComplete:
8789

8890
paths:
91+
# The Server acts as a "Smart Receiver" and implements a Seamless Fallback Pipeline (Zero-Drop)
92+
# by maintaining an ffmpeg -stream_loop on an /offline path (not shown here) during dropouts.
8993
~^wificam:
9094
~^cameraman:

0 commit comments

Comments
 (0)