Privately Owned Vehicle Work Group Meeting - 2025/04/28 - Slot 2

[m-zain-khawaja]

Agenda

• Update on EgoPath training
• Update on EgoLanes dataset curation
• Vision-Radar Fusion

---

EgoPath Network training

The full EgoPath training pipeline has been completed from the DataLoader class, Augmentations Class, Trainer Class and Main Train Loop - thank you to everyone for their efforts in helping achieve this. I did a basic PoC test of the EgoPath network on TuSimple dataset only and got positive results on just 40K training samples:

---

Example 1:

---

Example 2:

---

Example 3:

---

Loss Curve:

---

I also made a minor architectural change to the EgoPath head to remove DROPOUT from the very last to fully-connected layers to help the model converge better, as well as removing the SIGMOID layer on the output of the network to help gradient propagation through the network layers. I also tried a simple experiment to see whether directly predicting keypoints defining the EgoPath compared to a Bezier curve and I found that a Bezier curve representation converged much faster with much less noise compared to a keypoints based prediction.

Additionally, 4 loss functions were implemented:

1 - an endpoint loss which aligns the start and end control points of the ground truth and predicted bezier curves
2 - a mid-point loss, similar to the original BezierLaneNet paper, which attempts to align points along the curve
3 - a control point loss, which tries to align all 4 control points of the ground truth and predicted bezier curves
4 - a gradient loss, which acts as a regularizer and ensures that gradients match along the curve (there are 2 versions of this loss, an analytical loss and a numerical loss)

Furthermore, 3 options for batch size scheme were implemented:

1 - a constant batch size
2 - a fast decay of the batch size
3 - a slow decay of the batch size

@m-zain-khawaja to update the below issue with latest training options and add documentation in the README for example usage of the training script

• Network Training - EgoPath Network

---

EgoLanes Dataset Curation Update - Completed

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - 18,300 training data samples
• BDD100K - complted by @aadarshkt - PR has been merged with the main branch.

@TranHuuNhatHuy to create single data upload on Kaggle with the full EgoLanes dataset.
@m-zain-khawaja to work on initial EgoLanes network design

---

Vision-Radar Fusion

@m-zain-khawaja :

@docjag has been assigned the task to begin work on developing the Vision-Radar Fusion pipeline, beginning with fusing SceneSeg foreground object labels with Radar detections from automotive Radar. Details about this task can be found here

---

Attendees

• @m-zain-khawaja (Senior Tech Lead at AWF and Privately Owned Vehicle Work Group Chair)
• Mohammad Musa (Founder & CEO at Deepen AI and Safety Assurance Work Group Co-Chair)
• David Walmroth (US Director of Business and Project Development at PIX Moving and Co-Chair of Open AD Kit Work Group)
• Sunil Sheriker (Senior Director of Product at TierIV)
• Sergey Christ (Founder & CEO at Alternative Embedded)
• @TranHuuNhatHuy (Masters Robotics Student at Tokyo Institute of Technology)
• @docjag (Senior Computer Vision and Machine Learning Engineer)
• @dahhmani (Robotics Software Engineer at Vecna Robotics)
• @auo355 (Canada Revenue Agency, Information Technology Analyst/Developer)
• @pranavreddy23 (Graduate Student Researcher at NYU)
• Muhammad Ali (Undergraduate Teaching Assistant - Habib University)
• Srikanth Reddy Yalamakuru (Simulation and Solution Engineer at IPG Automotive)

Zoom Meeting Video Recording

Video Meeting Link
Please contact the work group lead (@m-zain-khawaja) to request access to a recording of this meeting.