[WIP] Object Map #60

avyan-k · 2026-01-18T00:51:28Z

Resolves #31 and #46. Uses YOLO model from #[YOLO_model_fitting PR].

Main Changes

Added Object Map ROS2 C++ Node
- the node uses the ZED SDK directly instead of the zed-ros2-wrapper, as the wrapper was found to have latency issues
Added zed_detection.hpp that collects all ZED SDK logic. This makes it easy to refactor out the ZED SDK if we decide to use the wrapper instead
Added a compilation definition (see HAS_ZED_SDK) to make the inclusion of the ZED SDK is optional. This should allow building of the vision package to work on machines without nvidia packages
Added object.hpp that defines DetectedObject, essentially a wrapper to the VisionObject from noetic base + humble base w ros1_bridge and custom_msgs #30 but with enum ObjectClass to avoid using string literals
Refactored vision package to support both C++ and Python nodes

Tests (to be) Done

[TBD] Test building on the jetson and see if ZED SDK is able to connect
[TBD] Determining necessary dependencies on the jetson (update Dockerfiles if needed)

Minor Changes

Changed VS Code editor default tab space to 4
Updated the VisionObject.msg definition to include an id field for improved object identification.

* Add: Composed CUDA Dockerfile without custom dependencies * Fixed compiling packages from source, proper mounting of ros2_ws and ZED_Explorer, will explain fix in more detail in PR * Delete Docker/jetson/Dockerfileold * Update docker-compose.yml * Update README.md * Update README.md * Added XSENS imu drivers along with mavros dependency WORK YAYAYYAYAYAY * Cleaned the dockerfile by removing useless commands and fusing colcon builds of dependencies, changed the README to update build command * Added the zed ros2 wrapper and Movella IMU drivers to repo * Add: CI pipeline for Docker images * Fix: Correct submodule path and dot file, Mod: workflow name for CI pipelines --------- Co-authored-by: rain1618 <[email protected]> Co-authored-by: Rain1618 <[email protected]> Co-authored-by: Alb33rt <[email protected]>

…witching inside the container

… pip to not timeout

…kerfile, will do back home cause stm strike haha

… and ultralytics, torch, torchvision fixed

…ped by the docker compose mount of the jetsons local AUV-2026 directory. also cleaned up some dependency installs

…nloading them in the notebook to better control and fix the errors outputted by some installs

…orkers needs to be tested on nano.

…UV-2027 into dev/YOLO_model_fitting

…_kf and update

…D_SDK, fallback to be added

…sted)

…mplement cam pose publishing (object map publishing is no-op for now)

…od signatures

…hing new the tracker interface

OwenLeSann and others added 30 commits September 22, 2025 13:19

Initialized model_pipeline

33ceb70

Debugged YOLO notebook. Should be ready to train.

5da2952

Added test set validation cell to notebook

d45c1e8

Reorganized file structure and added test set metrics cell to notebook

35f9ff5

"Changed name"

6ef4fbf

Cleaned file directory

cbb12de

Deleted clones in vision dir

d9bca7b

Fixed dev container rviz build error + mounting

9338171

added git clone in dockerfile and changed the mount to allow branch s…

ec441f4

…witching inside the container

Added env var to allow pip to not timeout

9fc120f

Added env var properly, using ENV var instead of RUN export, to allow…

5967954

… pip to not timeout

Fix: ultralytics pip issues, have to incorporate change into main doc…

528049d

…kerfile, will do back home cause stm strike haha

Fix: Combined all my dockerfile fixes into main dockerfile, pip fixed…

2c1e16b

… and ultralytics, torch, torchvision fixed

Removed the colcon build of xsens and zed wrapper as those will be wi…

29508f1

…ped by the docker compose mount of the jetsons local AUV-2026 directory. also cleaned up some dependency installs

Changed Dockerfile to have model pipeline dependencies instead of dow…

397fb9e

…nloading them in the notebook to better control and fix the errors outputted by some installs

Changed notebook API to access roboflow 2025 images.

a701ea4

Removed redundant arguments in model train function. Batch size and w…

b9677e9

…orkers needs to be tested on nano.

Removed redundant arguments in model train function. Batch size and w…

5f1c3ee

…orkers needs to be tested on nano.

Removed redundant arguments in model train function. Batch size and w…

36ffea6

…orkers needs to be tested on nano.

Resove merge

01b0b84

Removed hidden files.

2df88db

Removed hidden files.

c64f657

add yolov8n, yolov8s trained models

0829766

added yolov11m

d46b454

Merge branch 'dev/YOLO_model_fitting' of github.com:mcgill-robotics/A…

41bdcec

…UV-2027 into dev/YOLO_model_fitting

add roboflow data

d9780f1

attempt to upload data from yolov8.s model from Roboflow

41f1c1c

changed git ignore file to be able to upload jpg files

f9ed1ee

upload roboflow jpg files

8c74116

avyan-k and others added 25 commits November 24, 2025 10:38

Remove redundant enhancement testing script

7599ffd

prepare for enhancing dataset

7aac7f9

Merge branch 'dev/ros2_image_enhancement' into dev/YOLO_model_fitting

51f97db

add enhanced images

9c466c6

fix yolo training script

e7b07ab

Merge remote-tracking branch 'origin' into dev/YOLO_model_fitting

6051eef

fix docker

97dbc6c

Added scripts to test the SYTH unity dataset + removed imgs/labls git

365b897

Quick inference with relative 3d position test script (pure zedsdk)

fd7bd08

Quick testing of object map

848b94a

Prototyping Filters & Multi-Object Kalman Tracker (GNN + Mahalanobis)

047d1d9

Refactored for better clarity

773e15b

Merge remote-tracking branch 'origin/main' into dev/object_map

5f4a51a

Refactor vision package to include C++ nodes, add object map node

0326505

Set up object map for YOLO detection

cb864dd

Add: Kalman Filter and CMake Eigen3

0da208a

Add object_tracker header file def

c59c959

Separate ZED object detection logic into dedicated file

27d0e65

Refactor header: define Track struct and update signatures for create…

119f0cb

…_kf and update

extracted the logic of update() into private helper methods

61f6139

Complete Zed Detection (to be documented), Add compile options for ZE…

f62ad09

…D_SDK, fallback to be added

Add tentative fallback if ZED SDK or CUDA are not installed (to be te…

8d4372f

…sted)

Correct include logic based on whether ZED SDK is installed

0cd7d46

Make ZED detector use Eigen Mat instead of OpenCV Mat

e8b33d7

Add object tracker to map node, include publishing "interfaces" and i…

1c4e632

…mplement cam pose publishing (object map publishing is no-op for now)

avyan-k self-assigned this Jan 18, 2026

avyan-k linked an issue Jan 18, 2026 that may be closed by this pull request

Implement Object Map #46

Open

3 tasks

l-gzn added 3 commits January 20, 2026 05:31

add orientation and confidence fields to Track struct and update meth…

a16fb7d

…od signatures

update detector and node to pass orientation and confidence data matc…

4ad21cf

…hing new the tracker interface

add empty function definitions for tracking logic

aaa6c6b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] Object Map #60

[WIP] Object Map #60

Uh oh!

avyan-k commented Jan 18, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

[WIP] Object Map #60

Are you sure you want to change the base?

[WIP] Object Map #60

Uh oh!

Conversation

avyan-k commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Main Changes

Tests (to be) Done

Minor Changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

avyan-k commented Jan 18, 2026 •

edited

Loading