Skip to content

README.md

Latest commit

 

History

History
296 lines (222 loc) · 12.4 KB

README.md

File metadata and controls

296 lines (222 loc) · 12.4 KB
Raw

Spring-2026-Final-Project-Team11

This repository contains all information regarding the DonkeyCar Taxi for team 11.

RoboCar Taxi using Lidar, ROS2, and OAK-D Lite


Logo

ECE-MAE 148 Final Project

Team 11 Spring 2026


Table of Contents

  1. Team Members
  2. Abstract
  3. Promises
  4. Accomplishments
  5. Challenges
  6. Final Project Media
  7. Software
  8. Hardware
  9. Gantt Chart
  10. Course Deliverables
  11. Project Reproduction
  12. Acknowledgements
  13. Contacts

Team Members

Alan Ye - ECE LinkedIn

Wen Qiang - CS

Ryan Melson - MAE LinkedIn

Abstract

Our goal for the Final Project was to make a Taxi, that would bring a passenger, in our case a 3D printed duck, to a valid position to then stop for a short time and simulate a deboarding and boarding process. Using an enhance 2D LiDAR system coincided with an OAK-D Lite for visual confirmation and navigation. Valid parking being designated by a specific sign indicating temporary parking only.

Promises

  • Specific Object detection for allowed parking regions.
  • Object / Pedestrian detection using the LiDAR module.
  • Get the LiDAR unit operational into ROS2.

Accomplishments

  • Navigated a complex network within a Linux environment and ROS2 packages.
  • Implemented a YOLO model to identify and track custom 3D-printed parking signs.
  • Successfully integrated and configured a LiDAR sensor.

Future Work

If we were to redo this project in the future, it would have been necessary to spend more time learning and creating appropriate documentation on how to operate our LiDAR (see below for type). We would also need to learn how to implement it with an emergency stop feature and visually confirm passengers were on board. Visual confirmation being done with another OAK-D Lite camera with appropriate YOLO model. Being able to identify that it is at a stop and allowing the passenger to safely vacate the vehicle and the car understanding this as well.

Challenges

  • We ran into several component errors and issues leading us to have to delay our progress. As well as outside affairs we couldn't control.
  • LiDAR took a lot longer to learn and understand how it could be implemented into ROS2 than initially expected.
  • Time was our biggest struggle especially when only having 3 team members, we all had to take on additional roles to get to the stage we eventually got to.

Final Project Demo Videos

Media below shows what we were able to complete for our Final project.


**Sign Detection Demo**

Sign Detection Demo




**Web UI simulation**

Web UI simulation




**Outside view of robocar driving around compound**

Outside view of robocar driving around compound




**Slam Results**

Outside view of robocar driving around compound

Software --

Overall Architecture

Our project bridges a high-level web dispatch system with low-level ROS2 navigation. The central logic is split between a Python Flask server (handling the state machine and UI), an OAK-D vision script (handling spatial AI and passenger telemetry), and the ROS2 rclpy packages controlling the physical actuators.

Passenger UI & State Machine

The dispatch dashboard was built using HTML/CSS/JS and a Python Flask backend.

  • The State Machine: We utilized Python's threading library to run a continuous background physics/logic loop (update_vehicle_physics). This thread manages the active route, continuously calculating progress and transitioning between states (Idle, Arrived at Pickup, In Transit, Waiting for Disembark, and Emergency Stop).
  • Telemetry & Video Streaming: The dashboard polls the Flask API for JSON state updates to move the digital car along a circular CSS map. Simultaneously, it embeds a live MJPEG byte stream (multipart/x-mixed-replace) generated by the camera script, providing the passenger with a real-time, low-latency cabin view.

📁 web_dashboard/

  • app.py
    • Description: The main full-stack server application. It boots the Flask web framework, initializes the central vehicle state machine, spawns the non-blocking background simulation thread, and hosts the localized REST API endpoints.
    • How to Run: 
      cd web_dashboard
python app.py

OAK-D Spatial Vision & YOLOv8

We utilized the DepthAI pipeline to implement real-time passenger verification and station detection.

  • Custom YOLOv8 Training: We trained custom models to recognize our 3D-printed passenger duck and a custom parking sign, compiling them to run directly on the OAK-D's VPU.
  • Sensor Fusion: By overlaying the 2D YOLO bounding boxes onto the OAK-D's stereo depth map (YoloSpatialDetectionNetwork), we could extract the Z-axis distance in millimeters. Once the sign's distance falls below a 0.5m threshold, the state machine registers a station arrival.
  • Safety Interlocks: The camera script constantly verifies passenger presence. If the passenger is removed from the seat during transit, a hardware debouncing counter (preventing false-positives from blurs/shadows) trips, sending an API request to the central server that instantly forces the vehicle into an Emergency Stop.

📁 vision/

  • passenger_cam.py
    • Description: The core Edge AI pipeline script. It configures the OAK-D Lite camera pipeline, loads the compiled custom YOLOv8 .blob models onto the camera's VPU, handles the spatial stereo depth fusion math, tracks passenger presence with frame-debounce safety logic, and streams the MJPEG live feed to the local network.
    • How to Run: 
      cd vision
python passenger_cam.py
  • test_spatial_sign.py
    • Description: A dedicated hardware diagnostic and testing script used to validate the spatial coordinates and depth matching of the custom YOLOv8 traffic sign model prior to full integration.
    • How to Run: 
      cd vision
python test_spatial_sign.py

Obstacle Avoidance & mapping --

We used the LD06 Lidar to implement obstacle avoidance within ROS2. The program logic is quite simple in that we are constantly scanning the 60 degrees in front of the robot. If an object is detected within our distance threshold, the robot will accordingly make a turn to avoid it. Our logic for selecting which direction to turn in is quite simple in that if the object is on the left side, we first turn right, and otherwise, we turn left. Both turning directions include a corrective turn to bring the robot back to the centerline of the track and continue lane following.

we utilize the slam toolbox for mapping and Foxglove Studio to visualize the SLAM process

To start the SLAM First pull the docker image and download the Foxgolve Studio from https://foxglove.dev/download

docker pull wuweowo/robocar:latest

launch the corresponding file under

# for sick lidar
docker exec -it robocar_team11_joy bash
export ROS_DOMAIN_ID=11
source /opt/ros/foxy/setup.bash
source /home/projects/ros2_ws/install/setup.bash
ros2 launch ucsd_robocar_nav2_pkg all_components.launch.py

# for mapping and localization
docker exec -it robocar_team11_joy bash
export ROS_DOMAIN_ID=11
source /opt/ros/foxy/setup.bash
source /home/projects/ros2_ws/install/setup.bash
ros2 launch ucsd_robocar_nav2_pkg slam_launch.launch.py

# for joystick control
docker exec -it robocar_team11_joy bash
export ROS_DOMAIN_ID=11
source /opt/ros/foxy/setup.bash
source /home/projects/ros2_ws/install/setup.bash
ros2 launch ucsd_robocar_control2_pkg manual_joy_control_launch.launch.py

# save map
ros2 service call /slam_toolbox/save_map slam_toolbox/srv/SaveMap "{name: {data: '/home/projects/ros2_ws/src/ucsd_robocar_hub2/ucsd_robocar_nav2_pkg/ros_data/maps/my_map2'}}"

ros2 service call /slam_toolbox/serialize_map slam_toolbox/srv/SerializePoseGraph "{filename: 
  '/home/projects/ros2_ws/src/ucsd_robocar_hub2/ucsd_robocar_nav2_pkg/ros_data/maps/my_map2.posegraph'}"

# slam parameters
/home/projects/ros2_ws/install/ucsd_robocar_nav2_pkg/share/ucsd_robocar_nav2_pkg/config/slam_params_online_async.yaml

# lidar parameters
/home/projects/ros2_ws/install/ucsd_robocar_sensor2_pkg/share/ucsd_robocar_sensor2_pkg/config/sick_tim_5xx.yaml

# open port to host to establish rosbridge connection with foxglove
ssh -L 9090:127.0.0.1:9090 -N pi@ucsdrobocar-148-11.local

# change mode from mapping to localization 
nano /home/projects/ros2_ws/install/ucsd_robocar_nav2_pkg/share/ucsd_robocar_nav2_pkg/config/slam_params_online_async.yaml

'''
  mode: mapping -> mode: localization，
  add：
  map_file_name: /home/projects/ros2_ws/src/ucsd_robocar_hub2/ucsd_robocar_nav2_pkg/ros_data/maps/my_map
  map_start_at_dock: true
'''

home/donkey

Hardware

  • 3D Printing: All board mounts, Raspberry Pi Mount, Camera Mount and containment.
  • Laser Cutting: Base plate to mount electronics and other components.

Parts List

  • Traxxas Chassis with steering servo and sensored brushless DC motor
  • Servo PDB
  • Raspberry Pi
  • WiFi adapter
  • 64 GB Micro SD Card
  • Adapter/reader for Micro SD Card
  • Logitech F710 controller
  • OAK-D Lite Camera
  • SICK TiM 2D LiDAR
  • VESC
  • Anti-spark switch with power switch
  • DC-DC Converter
  • 4-cell LiPo battery
  • Battery voltage checker/alarm
  • DC Barrel Connector
  • XT60, XT30, MR60 connectors
  • USB-C to USB-A cable as well as USB-C to USB-C
  • Micro USB to USB cable

Baseplate

Rasperry Pi Case/Shell

Camera Mount

Circuit Diagram

Board Mounts

Parking Sign

Gantt Chart

References:

Contacts