Skip to content

inference.md

Latest commit

 

History

History
166 lines (115 loc) · 5.93 KB

inference.md

File metadata and controls

166 lines (115 loc) · 5.93 KB

Running on the Robot

🇷🇺 Русская версия

Deploying the trained model to Raspberry Pi and running sock detection from the camera.

Deploying to RPi

The recommended deployment path is described in launch.md and uses the built-in CLI:

./main.py deploy rpi5

Use this page after deployment to work with the web panel, local detection, and remote GPU inference.

Manual rsync fallback

If you want to copy the project manually, the fallback flow is:

rsync -avz --exclude .venv --exclude frontend/node_modules --exclude __pycache__ --exclude .git \
  ~/work/SocksTank/ rpi5:~/sockstank/

Web control panel (recommended)

Before using ssh rpi5 below, make sure the rpi5 host alias resolves on your development machine. If needed, configure ~/.ssh/config and /etc/hosts as described in infrastructure.md.

The main way to interact with the robot is through the web panel:

ssh rpi5
cd ~/sockstank
sudo -E nohup python3 main.py serve --model models/yolo11_best_ncnn_model --conf 0.5 > /tmp/sockstank.log 2>&1 &

Open in browser: http://rpi5:8080

The web panel includes: live video with YOLO detection, motor/servo/LED controls, telemetry (distance, IR sensors, CPU temperature).

sudo -E is required for camera and GPIO access (-E flag inherits user's PYTHONPATH).

Running detection (legacy)

Record video with detection to a file:

ssh rpi5
cd ~/sockstank
sudo -E python3 main.py detect --model models/yolo11_best_ncnn_model --conf 0.5

Parameters

Parameter Default Description
--model auto (.pt on dev/GPU, ncnn on RPi) Model path (.pt for GPU, ncnn directory for RPi)
--output detect.mp4 Output video file
--conf 0.5 Confidence threshold (0.0–1.0). Detections below the threshold are ignored
--frames 300 Maximum number of frames to record
--width 1280 Frame width (px)
--height 720 Frame height (px)
--fps 3 Camera frame rate

Example with ncnn model

If the model was exported to ncnn (see training):

sudo ./main.py detect --model models/yolo11_best_ncnn_model --conf 0.5  # NCNN for RPi

Remote Inference (GPU Server)

Inference can be offloaded to a remote GPU server (e.g., blackops with RTX 4070 SUPER — 314.8 FPS vs 14.9 FPS on RPi 5). The robot sends frames over HTTP, the server returns detections.

Inference Modes

In the web panel (http://rpi5:8080) under the Inference section there are three buttons:

Mode Description
auto Use GPU server if online, otherwise fallback to local
local Always use local inference (NCNN on RPi)
remote Always use remote inference (error if server is unavailable)

Quick rule of thumb:

  • use auto as the default day-to-day mode;
  • use local when debugging the RPi path or when the network/GPU host is unavailable;
  • use remote only when you explicitly want to force the GPU server.

Adding a GPU Server via UI

  1. Open web panel → Inference section in the right column
  2. Click + Add GPU Server
  3. Fill in the form:
    • Host — IP or hostname of the GPU server (e.g. 192.168.0.188)
    • Port — inference server port (default 8090)
    • Username — SSH user
    • Auth — SSH Key (path to key, default ~/.ssh/id_rsa) or Password
  4. Click Test to verify connection (shows GPU and model info)
  5. Click Save to save

The server appears in the list with a status indicator:

  • green — online
  • orange — starting
  • grey — offline

Start / Stop buttons launch and stop the inference server on the GPU host via SSH. The × button removes the server from the list.

Server configuration is saved in gpu_servers.json (in .gitignore).

Running via CLI

# On the GPU server (blackops)
cd ~/work/SocksTank
python3 -m server.inference_server --port 8090  # use python3 on Linux; auto-selects models/yolo11_best.pt on GPU/dev hosts

API

Method Endpoint Description
GET /api/inference Inference status (mode, backend, ms)
PUT /api/inference/mode Switch mode (auto/local/remote)
GET /api/gpu/servers List GPU servers
POST /api/gpu/servers Add GPU server
DELETE /api/gpu/servers/{host} Remove GPU server
POST /api/gpu/servers/{host}/test Test connection
POST /api/gpu/servers/{host}/start Start inference server
POST /api/gpu/servers/{host}/stop Stop inference server

How detection works

  1. The ov5647 camera captures frames via picamera2
  2. Each frame is passed to the YOLO model for sock detection
  3. Bounding boxes with class labels are drawn around detected socks
  4. Processed frames are written to the detect.mp4 video file

Viewing results

The detect.mp4 video file is saved in the current directory on the RPi. Copy it to your computer:

scp rpi5:~/sockstank/detect.mp4 .

Open with any video player (VLC, mpv, etc.).

Integration with tank controls

The robot is controlled via car.py from the Freenove kit. It supports several modes:

  • mode_ultrasonic — obstacle avoidance: ultrasonic sensor measures distance to objects. If < 45 cm — the robot reverses and turns
  • mode_infrared — line following: IR sensors track a line on the floor. When an object is detected at 5–12 cm — it's grabbed with the claw
  • mode_clamp — claw control: raise, lower, grab

The web panel (main.py serve) already combines detection with controls — you can drive the robot through the browser while seeing detection results in real time. Fully autonomous mode (find + drive + grab without human input) is the next development stage.

← Previous README Next →
Running the Project Back to README Inference Benchmarks