[Betatesting] Object Detection exercise

[javizqh]

Share here any issues, not solutions, related to this exercises. Do not comment anything else other than that.

This issue is not meant to be assigned to a single person, as its purpose is that multiple people test it.

[JitinSaxenaa]

Hi @javizqh

Beta-testing feedback:
I tested the Visual Object Detection exercise using a custom ONNX object detection model and verified real-time inference through the webcam pipeline. The exercise functions correctly and detections are rendered as expected.

The clear separation between model upload, inference pipeline, and WebGUI visualization, Made the workflow easy to understand and experiment with different models. I did not encounter any functional issues during testing.

[karttikjangid]

Beta-testing feedback – connectivity and documentation gap

While testing the Visual Object Detection exercise locally using the RoboticsAcademy Docker setup, I encountered a reproducible initialization issue related to the access URL and browser security context.

When the exercise is opened via http://0.0.0.0:7164:

• The WebSocket connection to ws://127.0.0.1:7163 repeatedly fails, preventing backend communication.
• Browser webcam access is blocked (getUserMedia error), leaving the camera panel without an image.
• As a result, the exercise appears non-functional even though the container and UI load correctly.

However, when accessing the same exercise through http://127.0.0.1:7164, the webcam stream initializes properly and the interface behaves as expected.
This indicates the failure is caused by browser secure-context/origin restrictions rather than the exercise logic itself.

Additionally, I could not find any mention in the current RoboticsAcademy user guide or exercise documentation that:

• the exercise depends on browser webcam access,
• it must be opened specifically via localhost/127.0.0.1,
• or that insecure origins like 0.0.0.0 may break WebSocket and camera initialization.

Because of this missing guidance, first-time users may interpret the behavior as an exercise failure instead of an access-context limitation.

I will continue testing the remaining execution stages and report any further issues found.

[karttikjangid]

@javizqh should i keep testing this exercise to find any other issues or move to other exercise .

[javizqh]

As you want. If you already think you have tested this exercise enough you may want to check another one. This issue will remain open, so that other people know that this exercise needs betatesting

[tony19053000]

Hello @javizqh,

My name is Aayush Kumar, and I tested the Visual Object Detection exercise using an ONNX human detection model.

I was able to get the pipeline working end-to-end. The model loads correctly, webcam input is processed, and human detection with bounding boxes and human count is working in the exercise.

While testing, I noticed one issue. The detection itself works correctly, but the output panel flickers during live inference. I also tried a lighter model for comparison. It was a bit more stable and the flickering was less, but it was still present.

This looks more like a performance/runtime or rendering issue during inference than a detection logic issue.

I am also attaching a demo video of my current working solution here:
Demo video

[prawnsgupta]

Hi @javizqh,

Beta-testing feedback — Visual Object Detection.

Tested on Windows 11 (Docker, CPU mode) with an SSD-MobileNet-v1 (COCO) ONNX model. Pipeline works end-to-end: model uploads, webcam feed processes, people get drawn as boxes with confidence scores in real time.

Issue 1 — misleading upload error. Uploading the .onnx model shows a red popup: "Error fetching file content: Reading ….onnx is not supported." Looks like the editor tries to text-preview the binary and fails. The upload actually succeeds and the exercise runs fine afterward — but this reads as a failure to a first-time user. Skipping preview for binary uploads (or a clearer message) would help.

Issue 2 — inference flicker in CPU mode. Box position and confidence jitter every frame. Measured in the UI: App: 30 Hz, Fps: 9, Lat: 102 ms — webcam captures ~20 fps but the model only processes ~9/sec, causing visible flicker. Feels like a CPU-mode performance characteristic, not a detection-logic bug — matches what another tester reported.

Note (not a bug): the model occasionally misclassified reflective objects (e.g. a metal bottle as "cell phone") — expected model-accuracy behavior, not a pipeline issue.

Access note: ran via http://localhost:7164 — webcam/WebSocket connected fine. Consistent with the earlier report that 0.0.0.0 breaks the secure-context requirements; might be worth stating this explicitly in the docs.

Environment: Windows 11, Docker Desktop (WSL2), CPU mode, onnxruntime 1.22, jderobot/robotics-academy:latest.

[jmplaza]

Hi @javizqh,

Beta-testing feedback — Visual Object Detection.

Tested on Windows 11 (Docker, CPU mode) with an SSD-MobileNet-v1 (COCO) ONNX model. Pipeline works end-to-end: model uploads, webcam feed processes, people get drawn as boxes with confidence scores in real time.

Issue 1 — misleading upload error. Uploading the .onnx model shows a red popup: "Error fetching file content: Reading ….onnx is not supported." Looks like the editor tries to text-preview the binary and fails. The upload actually succeeds and the exercise runs fine afterward — but this reads as a failure to a first-time user. Skipping preview for binary uploads (or a clearer message) would help.

Issue 2 — inference flicker in CPU mode. Box position and confidence jitter every frame. Measured in the UI: App: 30 Hz, Fps: 9, Lat: 102 ms — webcam captures ~20 fps but the model only processes ~9/sec, causing visible flicker. Feels like a CPU-mode performance characteristic, not a detection-logic bug — matches what another tester reported.

Note (not a bug): the model occasionally misclassified reflective objects (e.g. a metal bottle as "cell phone") — expected model-accuracy behavior, not a pipeline issue.

Access note: ran via http://localhost:7164 — webcam/WebSocket connected fine. Consistent with the earlier report that 0.0.0.0 breaks the secure-context requirements; might be worth stating this explicitly in the docs.

Environment: Windows 11, Docker Desktop (WSL2), CPU mode, onnxruntime 1.22, jderobot/robotics-academy:latest.

Very nice betatesting @prawnsgupta !