Skip to content

Commit fd83ee2

Browse files
authored
Merge branch 'main' into feat/59-backend-quality
2 parents 21d25e9 + 5d37755 commit fd83ee2

File tree

1 file changed

+44
-2
lines changed

1 file changed

+44
-2
lines changed

Documentation/README.md

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,47 @@ SPDX-FileCopyrightText: 2025 robot-visual-perception
44
SPDX-License-Identifier: CC-BY-4.0
55
-->
66

7-
Build, user, and technical documentation
8-
Software architecture description
7+
## Build, User and Technical Documentation
8+
9+
TODO
10+
11+
## Software architecture description
12+
13+
### Frontend (React + TypeScript)
14+
15+
- Displays the live video feed received through WebRTC.
16+
- Connects to the backend via WebSocket to receive image.analysis results as metadata.
17+
- Renders bounding boxes and distance overlays in real time.
18+
19+
### Backend (Python + FastAPI)
20+
21+
- Handles WebRTC video streaming using the aiortc library.
22+
- Manages WebRTC signaling process.
23+
24+
#### Image Analysis Service (Python + Pytorch + OpenCV)
25+
26+
- Processes video frames to detect and locate objects in 3D space using ultralytics' YOLO models.
27+
- Uses monocular depth estimation utilizing AI-Models like MiDas for distance calculation.
28+
- Outputs object metadata (bounding boxes, labels, confidence, distance).
29+
- Sends results to Frontend via Websocket.
30+
31+
### Infrastructure
32+
33+
- All components are containerized with Docker and will be orchestrated with Kubernets.
34+
- Horizontal scaling is supported based on GPU availability and number of concurrent WebSocket clients.
35+
- Logging and monitoring are provided through Prometheus and Grafana integrations.
36+
37+
### Design Rationale
38+
39+
The choice of WebSocket for metadata streaming ensures low-latency, browser-native communication [1] without the complexity of WebRTC DataChannels, which are faster in general but much harder to deal with in (future) cloud environments like Kubernetes, since it uses UDP Peer-to-Peer connections [2][3]. Load balancing in this case is also not trivial [4].
40+
41+
### References
42+
43+
1. Ably, "WebSockets explained: What they are and how they work", [link](https://ably.com/topic/websockets)
44+
45+
2. VideoSDK, "WebSockets vs WebRTC: Key Differences and Best Use Cases Explained", [link](https://www.videosdk.live/developer-hub/developer-hub/webrtc/websockets-vs-webrtc-differences)
46+
47+
3. Medium, "Kubernetes: The next step for WebRTC", [link](https://medium.com/l7mp-technologies/kubernetes-the-next-step-for-webrtc-fb8d8a33f24e)
48+
49+
4. ossrs, "Load Balancing Streaming Servers", [link](https://ossrs.net/lts/en-us/blog/load-balancing-streaming-servers)
50+

0 commit comments

Comments
 (0)