Tele Imager

English | 中文

1. Image Server

This repository provides an image server that captures video streams from multiple cameras (UVC, OpenCV, and RealSense) and publishes them over the network using ZeroMQ or WebRTC.

Currently, Tele Imager is used in the xr_teleoperate project to provide teleoperation video streams.

All user-callable APIs are located under the # public api comment in the code.

1.0 ✨ Features

📸 Supports multiple UVC, OpenCV, and Intel RealSense cameras
📢 Publishes video frames using ZeroMQ PUB-SUB
📢 Publishes video frames using WebRTC
🚧 (TODO) Local shared memory mode for ultra-low latency frame access
💬 Responds to image configuration commands via ZeroMQ REQ-REP
🆔 Multiple camera identifiers: physical path, serial number, or video device path
⚙️ Configurable resolution and frame rate
🚀 Efficient frame handling using a triple ring buffer

1.1 📥 Environment Setup

Install miniconda3:

# for Jetson Orin NX (ARM architecture)
unitree@ubuntu:~$ mkdir -p ~/miniconda3
unitree@ubuntu:~$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O ~/miniconda3/miniconda.sh
unitree@ubuntu:~$ bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
unitree@ubuntu:~$ rm ~/miniconda3/miniconda.sh
unitree@ubuntu:~$ source ~/miniconda3/bin/activate
(base) unitree@ubuntu:~$ conda init --all

Create and activate a conda environment:

(base) unitree@ubuntu:~$ conda create -n teleimager python=3.10 -y
(base) unitree@ubuntu:~$ conda activate teleimager

Install the repository and dependencies:

(teleimager) unitree@ubuntu:~$ sudo apt install -y libusb-1.0-0-dev libturbojpeg-dev
(teleimager) unitree@ubuntu:~$ git clone https://github.com/unitreerobotics/teleimager.git
(teleimager) unitree@ubuntu:~$ cd teleimager
# If you only use the client
(teleimager) unitree@ubuntu:~/teleimager$ pip install -e .
# If you also use the server
(teleimager) unitree@ubuntu:~/teleimager$ pip install -e ".[server]"

Add video device permissions for non-root users:

bash setup_uvc.sh

Configure certificate paths (required for WebRTC):

Certificates are usually generated by televuer. You can specify the certificate location via user config directory or environment variables.

Method 1: User config directory (recommended)
```
mkdir -p ~/.config/xr_teleoperate/
cp cert.pem key.pem ~/.config/xr_teleoperate/
```
Method 2: Environment variables
```
echo 'export XR_TELEOP_CERT="your_file_path/cert.pem"' >> ~/.bashrc
echo 'export XR_TELEOP_KEY="your_file_path/key.pem"' >> ~/.bashrc
source ~/.bashrc
```
Method 3: Default behavior

If neither method is used, Tele Imager will look for certificates in the default module paths.

1.2 🔍 Discover Connected Cameras

Run the following command to discover connected cameras:

python -m teleimager.image_server --cf
# or
teleimager-server --cf

You will see output similar to:

(teleimager) unitree@ubuntu:~$ python -m teleimager.image_server --cf
10:24:35:849900 INFO     ======================= Camera Discovery Start ================================== image_server.py:216
10:24:35:851008 INFO     Found video devices: ['/dev/video0', '/dev/video1', '/dev/video2', '/dev/video3', image_server.py:217
                         '/dev/video4', '/dev/video5']                                                                        
10:24:35:852089 INFO     Found RGB video devices: ['/dev/video0', '/dev/video2', '/dev/video4']            image_server.py:218
10:24:35:852280 INFO     ------------------------- UVC Camera 1 ------------------------------------       image_server.py:227
10:24:35:852575 INFO     video_path    : /dev/video0                                                       image_server.py:228
10:24:35:852759 INFO     video_id      : 0                                                                 image_server.py:229
10:24:35:852844 INFO     serial_number : 200901010002                                                      image_server.py:230
10:24:35:852919 INFO     physical_path : /sys/devices/pci0000:00/0000:00:14.0/usb1/1-5/1-5:1.0             image_server.py:231
10:24:35:852989 INFO     extra_info:                                                                       image_server.py:232
10:24:35:853062 INFO         name: USB HDR Camera                                                          image_server.py:239
10:24:35:853133 INFO         manufacturer: Generic                                                         image_server.py:239
10:24:35:853198 INFO         serialNumber: 200901010002                                                    image_server.py:239
10:24:35:853261 INFO         idProduct: 8272                                                               image_server.py:239
10:24:35:853336 INFO         idVendor: 7749                                                                image_server.py:239
10:24:35:853399 INFO         device_address: 4                                                             image_server.py:239
10:24:35:853735 INFO         bus_number: 1                                                                 image_server.py:239
10:24:35:853829 INFO         uid: 1:4                                                                      image_server.py:239
...
10:24:36:033234 INFO         format: 480x640@30 MJPG                                                       image_server.py:243
10:24:36:033249 INFO         format: 480x640@60 MJPG                                                       image_server.py:243
...
10:24:36:034519 INFO     ------------------------- UVC Camera 2 ------------------------------------       image_server.py:227
10:24:36:034551 INFO     video_path    : /dev/video2                                                       image_server.py:228
10:24:36:034567 INFO     video_id      : 2                                                                 image_server.py:229
10:24:36:034582 INFO     serial_number : 01.00.00                                                          image_server.py:230
10:24:36:034595 INFO     physical_path : /sys/devices/pci0000:00/0000:00:14.0/usb1/1-11/1-11.1/1-11.1:1.0  image_server.py:231
10:24:36:034608 INFO     extra_info:                                                                       image_server.py:232
10:24:36:034622 INFO         name: Cherry Dual Camera                                                      image_server.py:239
10:24:36:034635 INFO         manufacturer: DECXIN                                                          image_server.py:239
10:24:36:034647 INFO         serialNumber: 01.00.00                                                        image_server.py:239
10:24:36:034658 INFO         idProduct: 11599                                                              image_server.py:239
10:24:36:034670 INFO         idVendor: 7119                                                                image_server.py:239
10:24:36:034683 INFO         device_address: 9                                                             image_server.py:239
10:24:36:034695 INFO         bus_number: 1                                                                 image_server.py:239
10:24:36:034710 INFO         uid: 1:9                                                                      image_server.py:239
...
10:24:36:435928 INFO         format: 480x1280@10 MJPG                                                      image_server.py:243
10:24:36:435988 INFO         format: 480x1280@15 MJPG                                                      image_server.py:243
10:24:36:436047 INFO         format: 480x1280@20 MJPG                                                      image_server.py:243
10:24:36:436108 INFO         format: 480x1280@25 MJPG                                                      image_server.py:243
10:24:36:436168 INFO         format: 480x1280@30 MJPG                                                      image_server.py:243
10:24:36:436227 INFO         format: 480x1280@60 MJPG                                                      image_server.py:243
10:24:36:436286 INFO         format: 480x1280@120 MJPG                                                     image_server.py:243
...
10:24:36:524038 INFO     ------------------------- UVC Camera 3 ------------------------------------       image_server.py:227
10:24:36:524203 INFO     video_path    : /dev/video4                                                       image_server.py:228
10:24:36:524282 INFO     video_id      : 4                                                                 image_server.py:229
10:24:36:524345 INFO     serial_number : 200901010001                                                      image_server.py:230
10:24:36:524398 INFO     physical_path : /sys/devices/pci0000:00/0000:00:14.0/usb1/1-11/1-11.2/1-11.2:1.0  image_server.py:231
10:24:36:524449 INFO     extra_info:                                                                       image_server.py:232
10:24:36:524531 INFO         name: USB HDR Camera                                                          image_server.py:239
10:24:36:524672 INFO         manufacturer: Generic                                                         image_server.py:239
10:24:36:524734 INFO         serialNumber: 200901010001                                                    image_server.py:239
10:24:36:524789 INFO         idProduct: 8272                                                               image_server.py:239
10:24:36:524843 INFO         idVendor: 7749                                                                image_server.py:239
10:24:36:524893 INFO         device_address: 10                                                            image_server.py:239
10:24:36:524942 INFO         bus_number: 1                                                                 image_server.py:239
10:24:36:524989 INFO         uid: 1:10                                                                     image_server.py:239
10:24:36:688311 INFO         format: 240x320@30 MJPG                                                       image_server.py:243
...
10:24:36:689031 INFO         format: 480x640@30 MJPG                                                       image_server.py:243
10:24:36:689089 INFO         format: 480x640@60 MJPG                                                       image_server.py:243
...
10:24:36:714374 INFO     =========================== Camera Discovery End ================================

If RealSense cameras exist, add the --rs argument to see RealSense discovery results.

1.3 📡 Start the Image Server

After filling cam_config_server.yaml according to camera discovery, start the server:

python -m teleimager.image_server
python -m teleimager.image_server --rs   # if using RealSense

# or
teleimager-server
teleimager-server --rs

2. Image Client

The client module connects to the image server to receive and display multiple video streams. Designed for teleoperation scenarios.

All user-callable APIs are under # public api.

2.1 🌀 Using ZMQ

After the server is running, start the client in another terminal:

python -m teleimager.image_client
# or
teleimager-client --host 127.0.0.1

If the server runs on a NVIDIA Jetson with IP 192.168.123.164:

teleimager-client --host 192.168.123.164

You will see separate OpenCV windows showing each camera stream.

Ensure opencv-python is installed.

2.2 🌀 Using WebRTC

For WebRTC streams, open a browser:

https://<host_ip>:<webrtc_port>
# e.g.
https://192.168.123.164:60001

And please press start button top-right.

3. 🚀 Automatic Startup Service

After successful setup and testing, enable automatic startup:

bash setup_autostart.sh

Follow the prompts to complete configuration.

4. 🧠 Design Principles

4.1 Why Support Multiple Camera Identifiers?

Linux cameras can be identified in multiple ways:

Physical path (physical_path)
Serial number (serial_number)
Video device path (video_id → /dev/videoX)

Each has pros and cons. Tele Imager supports all three for stable and flexible identification.

1. Physical path

🎯 Advantages:

Not affected by reboot or plug order
Works even if cameras share serial numbers
Ideal for fixed multi-camera deployments (robot head + wrists)

⚠️ Disadvantages:

Not flexible: changing USB port requires config update

2. Serial number

🎯 Advantages:

Remains constant across USB ports
High identification accuracy
Simple configuration
Recommended for RealSense

⚠️ Disadvantages:

Low-cost cameras may reuse serial numbers
Some serial numbers are unstable or malformed

3. Video device path (/dev/videoX)

🎯 Advantages:

Direct usage: just specify video_id: X
Good for single camera or temporary testing

⚠️ Disadvantages:

Changes with plug order or reboot
Unreliable for multi-camera setups

Identifier	Stability	Flexibility	Recommended Use
Physical path	⭐⭐⭐⭐⭐	⭐	Multi-camera fixed setup, low-cost cameras
Serial number	⭐⭐⭐⭐	⭐⭐⭐	RealSense, unique serial cameras
video_id	⭐⭐	⭐⭐⭐⭐⭐	Single camera, temporary testing

4.2 Why Support Three Transmission Methods?

The image server has two main uses:

Recording high-quality data → For model training
Real-time visualization (XR/UI) → For debugging and monitoring

Different scenarios (local, LAN, remote) require different latency and bandwidth trade-offs. Thus, three transmission methods are supported.

1. ZeroMQ PUB–SUB

Use case: Server and client on different machines over LAN

🎯 Advantages

High-quality frame transmission in LAN
Low overhead, high throughput
Low latency without sacrificing image quality

2. WebRTC

Use case: Real-time preview, VR teleoperation, UI debugging

🎯 Advantages

Low latency with adaptive bitrate
H.264(default) / VP8
Browser and VR device compatible

3. Shared Memory

Use case: Server and client on same machine for maximum performance

🎯 Advantages

Maximum bandwidth (memory-limited)
μs-level latency
Zero-copy or single-copy possible
Low CPU usage

4.3 Triple Ring Buffer Benefits

Non-blocking Read/Write:

Writer does not wait for reader; keeps writing in available slots

Reader always fetches the latest complete frame
No Tearing:

Reader never reads a partially written frame due to write avoidance logic
Always Fresh:

Unlike FIFO queues, old frames can be overwritten, so the reader always gets the latest frame

Critical for real-time applications

5. 🧐 FAQ

Why is the serial number and other information displayed as "unknown" in the teleimager-server --cf output?

You can try running the command with sudo privileges. Some cameras require elevated permissions to retrieve full hardware metadata. For example:
```
sudo $(which teleimager-server) --cf
```

6. 🙏 Acknowledgement

Some code references: https://github.com/ARCLab-MIT/beavr-bot

Name		Name	Last commit message	Last commit date
Latest commit History 46 Commits
src/teleimager		src/teleimager
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
README_zh-CN.md		README_zh-CN.md
cam_config_server.yaml		cam_config_server.yaml
pyproject.toml		pyproject.toml
setup_autostart.sh		setup_autostart.sh
setup_uvc.sh		setup_uvc.sh

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

Tele Imager

1. Image Server

1.0 ✨ Features

1.1 📥 Environment Setup

1.2 🔍 Discover Connected Cameras

1.3 📡 Start the Image Server

2. Image Client

2.1 🌀 Using ZMQ

2.2 🌀 Using WebRTC

3. 🚀 Automatic Startup Service

4. 🧠 Design Principles

4.1 Why Support Multiple Camera Identifiers?

4.2 Why Support Three Transmission Methods?

4.3 Triple Ring Buffer Benefits

5. 🧐 FAQ

6. 🙏 Acknowledgement

About

Uh oh!

Releases

Packages

Languages

License

unitreerobotics/teleimager

Folders and files

Latest commit

History

Repository files navigation

Tele Imager

1. Image Server

1.0 ✨ Features

1.1 📥 Environment Setup

1.2 🔍 Discover Connected Cameras

1.3 📡 Start the Image Server

2. Image Client

2.1 🌀 Using ZMQ

2.2 🌀 Using WebRTC

3. 🚀 Automatic Startup Service

4. 🧠 Design Principles

4.1 Why Support Multiple Camera Identifiers?

4.2 Why Support Three Transmission Methods?

4.3 Triple Ring Buffer Benefits

5. 🧐 FAQ

6. 🙏 Acknowledgement

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages