Feat/add zmq camera #2563

nepyope · 2025-12-01T15:45:16Z

add zmq camera, should allow for lerobot teleoperate / record / replay.

Copilot

Pull request overview

This PR adds ZMQ (ZeroMQ) camera support to the lerobot framework, enabling remote camera streaming over network connections. The implementation includes a new ZMQCamera class with support for multiple message formats (msgpack, JSON, and raw JPEG), automatic camera discovery via network scanning, and both synchronous and asynchronous frame reading capabilities. However, the PR contains several critical bugs that will prevent the code from running:

Critical Issues:

All script files attempt to import and use a non-existent busy_wait function instead of the existing precise_sleep function, causing ImportError
The custom teleoperator import doesn't exist and will cause ImportError
ZMQ socket thread-safety issue where the same socket is used from multiple threads

Key Changes:

Added complete ZMQ camera implementation with multi-format support and network scanning
Updated recording, replay, and teleoperation scripts to support ZMQ cameras
Modified recording behavior to support neutral/idle actions when no teleoperator or policy is provided

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 25 comments.

Show a summary per file

File	Description
`src/lerobot/cameras/zmq/__init__.py`	Module initialization for ZMQ camera package
`src/lerobot/cameras/zmq/configuration_zmq.py`	Configuration class for ZMQ cameras with validation
`src/lerobot/cameras/zmq/camera_zmq.py`	Core ZMQ camera implementation with network discovery and async reading
`src/lerobot/scripts/lerobot_teleoperate.py`	Added ZMQ camera support and unitree_g1 robot; changed to non-existent `busy_wait` function (critical bug)
`src/lerobot/scripts/lerobot_replay.py`	Added unitree_g1 robot support; changed to non-existent `busy_wait` function (critical bug)
`src/lerobot/scripts/lerobot_record.py`	Added ZMQ camera, unitree_g1 robot, neutral action mode; changed to non-existent `busy_wait` function (critical bug)
`src/lerobot/scripts/lerobot_find_cameras.py`	Extended camera discovery to include ZMQ cameras with network scanning

Comments suppressed due to low confidence (9)

src/lerobot/cameras/zmq/camera_zmq.py:321

Multiple bare except clauses (lines 309-310, 320-321). Consider catching specific exception types or at minimum use except Exception to avoid masking system-exiting exceptions.

                                except:
                                    pass
                except:
                    pass
            
            # Raw JPEG
            if frame is None:
                try:
                    frame = cv2.imdecode(np.frombuffer(msg, np.uint8), cv2.IMREAD_COLOR)
                    if frame is not None:
                        fmt = "raw_jpeg"
                except:
                    pass

src/lerobot/cameras/zmq/camera_zmq.py:294

Except block directly handles BaseException.

            except:

src/lerobot/cameras/zmq/camera_zmq.py:309

Except block directly handles BaseException.

                                except:

src/lerobot/cameras/zmq/camera_zmq.py:311

Except block directly handles BaseException.

                except:

src/lerobot/cameras/zmq/camera_zmq.py:320

Except block directly handles BaseException.

                except:

src/lerobot/cameras/zmq/camera_zmq.py:294

'except' clause does nothing but pass and there is no explanatory comment.

            except:

src/lerobot/cameras/zmq/camera_zmq.py:309

'except' clause does nothing but pass and there is no explanatory comment.

                                except:

src/lerobot/cameras/zmq/camera_zmq.py:311

'except' clause does nothing but pass and there is no explanatory comment.

                except:

src/lerobot/cameras/zmq/camera_zmq.py:320

'except' clause does nothing but pass and there is no explanatory comment.

                except:

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

Copilot · 2025-12-01T15:55:45Z

src/lerobot/scripts/lerobot_replay.py


        dt_s = time.perf_counter() - start_episode_t
-        precise_sleep(1 / dataset.fps - dt_s)
+        busy_wait(1 / dataset.fps - dt_s)


The call to busy_wait(1 / dataset.fps - dt_s) will fail because the busy_wait function doesn't exist in lerobot.utils.robot_utils. The correct function name is precise_sleep.

Copilot · 2025-12-01T15:55:45Z

src/lerobot/scripts/lerobot_record.py


        dt_s = time.perf_counter() - start_loop_t
-        precise_sleep(1 / fps - dt_s)
+        busy_wait(1 / fps - dt_s)


The call to busy_wait(1 / fps - dt_s) will fail because the busy_wait function doesn't exist in lerobot.utils.robot_utils. The correct function name is precise_sleep.

Copilot · 2025-12-01T15:55:46Z

src/lerobot/cameras/zmq/camera_zmq.py

+                except:
+                    pass


Bare except clause. Consider catching specific exception types or at minimum use except Exception to avoid masking system-exiting exceptions.

Copilot · 2025-12-01T15:55:46Z

src/lerobot/cameras/zmq/camera_zmq.py

+        def test_target(host_ip: str, port: int) -> dict | None:
+            """Test one host:port for ZMQ camera."""
+            ctx = zmq.Context()
+            sock = ctx.socket(zmq.SUB)
+            sock.connect(f"tcp://{host_ip}:{port}")
+            sock.setsockopt_string(zmq.SUBSCRIBE, "")
+            sock.setsockopt(zmq.RCVTIMEO, timeout_ms)
+
+            # Wait for subscription to establish (ZMQ "slow joiner" problem)
+            time.sleep(0.1)
+
+            # Try receiving a few times
+            msg = None
+            for _ in range(3):
+                try:
+                    msg = sock.recv()
+                    break
+                except zmq.Again:
+                    time.sleep(0.05)
+
+            if msg is None:
+                sock.close()
+                ctx.term()
+                return None


[nitpick] The test_target function uses hard-coded small delays and retry logic (lines 264, 273) without clear timeout bounds. While the individual socket has timeout_ms, the overall function could hang longer than expected when dealing with slow or partially-responsive servers. Consider adding an overall timeout mechanism or making the retry logic more explicit.

Copilot · 2025-12-01T15:55:46Z

src/lerobot/cameras/zmq/camera_zmq.py

+
+        if frame is None:
+            raise RuntimeError(f"Internal error: Event set but no frame available for {self}.")
+


[nitpick] Potential race condition: The latest_frame is accessed without holding the lock on line 592 (in the check if frame is None), but it was read with the lock on lines 588-590. While this is likely harmless in practice (since frame is a local copy), the check on line 592 should logically be inside the with self.frame_lock: block for consistency and to avoid any potential issues with Python's memory model.

Suggested change

if frame is None:

raise RuntimeError(f"Internal error: Event set but no frame available for {self}.")

if frame is None:

raise RuntimeError(f"Internal error: Event set but no frame available for {self}.")

Copilot · 2025-12-01T15:55:51Z

src/lerobot/scripts/lerobot_teleoperate.py

    make_teleoperator_from_config,
    so100_leader,
    so101_leader,
+    custom


[nitpick] Missing comma at the end of the import. This line should end with a comma for consistency with Python import formatting conventions when imports span multiple lines.

Suggested change

custom

custom,

Copilot · 2025-12-01T15:55:52Z

src/lerobot/scripts/lerobot_record.py

    make_teleoperator_from_config,
    so100_leader,
    so101_leader,
+    custom


[nitpick] Missing comma at the end of the import. This line should end with a comma for consistency with Python import formatting conventions when imports span multiple lines.

Suggested change

custom

custom,

Copilot · 2025-12-01T15:55:52Z

src/lerobot/cameras/zmq/camera_zmq.py

+
+import json
+import logging
+import os


Import of 'os' is not used.

Suggested change

import os

Copilot · 2025-12-01T15:55:52Z

src/lerobot/cameras/zmq/camera_zmq.py

+import json
+import logging
+import os
+import threading


Import of 'threading' is not used.

Suggested change

import threading

Copilot · 2025-12-01T15:55:53Z

src/lerobot/cameras/zmq/camera_zmq.py

+import os
+import threading
+import time
+from pathlib import Path


Import of 'Path' is not used.

Suggested change

from pathlib import Path

nepyope added 3 commits December 1, 2025 16:25

add ZMQ camera class

6b77224

modify teleoperate/replay/record to accomodate for g1

883e623

modify teleoperate/replay/record to accomodate for g1

44fbaaa

Copilot AI review requested due to automatic review settings December 1, 2025 15:45

Copilot started reviewing on behalf of nepyope December 1, 2025 15:45 View session

Copilot finished reviewing on behalf of nepyope December 1, 2025 15:51

Copilot AI reviewed Dec 1, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feat/add zmq camera #2563

Feat/add zmq camera #2563

Uh oh!

nepyope commented Dec 1, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		if frame is None:
		raise RuntimeError(f"Internal error: Event set but no frame available for {self}.")

Feat/add zmq camera #2563

Are you sure you want to change the base?

Feat/add zmq camera #2563

Uh oh!

Conversation

nepyope commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nepyope commented Dec 1, 2025 •

edited

Loading