|
1 |
| -- title: Safe and Robust Interaction-Aware Decision Making for Human-Robot Interactions |
| 1 | +- title: Safe, Efficient, and Robust Decision Making for Human-Robot Interactions |
2 | 2 | # subtitle: a subtitle
|
3 | 3 | # group: featured
|
4 | 4 | image: research_images/human_robot_interaction.png
|
|
32 | 32 | We investigate foundation models and vision language models (VLMs) for robotics and autonomous systems to enhance their reasoning capability and reliability.
|
33 | 33 | For example, inferring the short-term and long-term intentions of traffic participants and understanding the contextual semantics of scenes are the keys to scene understanding and situational awareness of autonomous vehicles.
|
34 | 34 | Moreover, how to enable autonomous agents (e.g., self-driving cars) to explain their reasoning, prediction, and decision making processes to human users (e.g., drivers, passengers) in a human understandable form (e.g., natural language) to build humans’ trust remains largely underexplored.
|
35 |
| - Therefore, **we created the first multimodal dataset for a new risk object ranking and natural language explanation task in urban scenarios and a rich dataset for intention prediction in autonomous driving, establishing benchmarks for corresponding tasks. Meanwhile, our research introduced novel methods that achieve superior performance on these problems.**<br><br> |
| 35 | + Therefore, **we created the first multimodal dataset for a new risk object ranking and natural language explanation task in urban scenarios and a rich dataset for intention prediction in autonomous driving, establishing benchmarks for corresponding tasks. Meanwhile, our research introduced novel methods that achieve superior performance on these problems.** |
| 36 | + Moreover, we explore effective solutions for complex long-horizon robotic tasks and multi-agent collaborations with multi-modal foundation models. <br><br> |
36 | 37 |
|
37 | 38 | **Related Publications\:** <br>
|
38 |
| - 1. [LOKI: Long Term and Key Intentions for Trajectory Prediction](https://openaccess.thecvf.com/content/ICCV2021/html/Girase_LOKI_Long_Term_and_Key_Intentions_for_Trajectory_Prediction_ICCV_2021_paper.html), ICCV 2021. <br> |
39 |
| - 2. [Important Object Identification with Semi-Supervised Learning for Autonomous Driving](https://arxiv.org/abs/2203.02634), ICRA 2022. <br> |
40 |
| - 3. [DRAMA: Joint Risk Localization and Captioning in Driving](https://arxiv.org/abs/2209.10767), WACV 2023. <br> |
41 |
| - 4. [Rank2Tell: A Multimodal Dataset for Joint Driving Importance Ranking and Reasoning](https://arxiv.org/abs/2309.06597), WACV 2024. |
| 39 | + 1. [Rank2Tell: A Multimodal Dataset for Joint Driving Importance Ranking and Reasoning](https://arxiv.org/abs/2309.06597), WACV 2024. <br> |
| 40 | + 2. [DRAMA: Joint Risk Localization and Captioning in Driving](https://arxiv.org/abs/2209.10767), WACV 2023. <br> |
| 41 | + 3. [Important Object Identification with Semi-Supervised Learning for Autonomous Driving](https://arxiv.org/abs/2203.02634), ICRA 2022. <br> |
| 42 | + 4. [LOKI: Long Term and Key Intentions for Trajectory Prediction](https://openaccess.thecvf.com/content/ICCV2021/html/Girase_LOKI_Long_Term_and_Key_Intentions_for_Trajectory_Prediction_ICCV_2021_paper.html), ICCV 2021. |
42 | 43 | # repo: greenelab/lab-website-template
|
43 | 44 | tags:
|
44 | 45 | - Foundation Models
|
|
49 | 50 | - Cooperative Planning
|
50 | 51 |
|
51 | 52 |
|
52 |
| -- title: Explainable and Generalizable Relational Reasoning and Multi-Agent Interaction Modeling (Social & Physical) |
| 53 | +- title: Explainable and Generalizable Relational Reasoning and Multi-Agent Interaction Modeling |
53 | 54 | # subtitle: a subtitle
|
54 | 55 | # group: featured
|
55 | 56 | image: research_images/Explainable Relational Reasoning.webp
|
|
61 | 62 | **We also proposed a physics-guided relational learning approach for physical dynamics modeling.**<br><br>
|
62 | 63 |
|
63 | 64 | **Related Publications\:** <br>
|
64 |
| - 1. [Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation](https://arxiv.org/abs/2401.12275), submitted to IEEE Transactions on Robotics (T-RO). <br> |
| 65 | + 1. [Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation](https://arxiv.org/abs/2401.12275), submitted to IEEE Transactions on Robotics (T-RO), under review. <br> |
65 | 66 | 2. [Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Crowd Trajectory Forecasting](https://arxiv.org/abs/2109.14128), ICRA 2022. <br>
|
66 | 67 | 3. [Learning Physical Dynamics with Subequivariant Graph Neural Networks](https://proceedings.neurips.cc/paper_files/paper/2022/hash/a845fdc3f87751710218718adb634fe7-Abstract-Conference.html), NeurIPS 2022. <br>
|
67 | 68 | 4. [Interaction Modeling with Multiplex Attention](https://proceedings.neurips.cc/paper_files/paper/2022/hash/7e6361a5d73a8fab093dd8453e0b106f-Abstract-Conference.html), NeurIPS 2022. <br>
|
|
83 | 84 |
|
84 | 85 |
|
85 | 86 |
|
| 87 | +- title: Generalizable and Diverse Trajectory and Occupancy Prediction for Autonomous Driving |
| 88 | + # subtitle: a subtitle |
| 89 | + # group: featured |
| 90 | + image: research_images/trajectory_occupancy_prediction.png |
| 91 | + # link: https://github.com/ |
| 92 | + description: | |
| 93 | + We <br><br> |
| 94 | + |
| 95 | + **Related Publications\:** <br> |
| 96 | + 1. [CMP: Cooperative Motion Prediction with Multi-Agent Communication](https://arxiv.org/abs/2403.17916), submitted to IEEE Robotics and Automation Letters (RA-L), under review. <br> |
| 97 | + 2. [Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting](https://arxiv.org/abs/2407.09475), submitted to IEEE Robotics and Automation Letters (RA-L), under review. <br> |
| 98 | + 3. [Self-Supervised Multi-Future Occupancy Forecasting for Autonomous Driving](https://arxiv.org/abs/2407.21126), submitted to IEEE Robotics and Automation Letters (RA-L), under review. <br> |
| 99 | + 4. [Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments](https://arxiv.org/abs/2309.13893), ICRA 2024. <br> |
| 100 | + 5. [Predicting Future Spatiotemporal Occupancy Grids with Semantics for Autonomous Driving](https://arxiv.org/abs/2310.01723), IV 2024. <br> |
| 101 | + 6. [Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments](https://arxiv.org/abs/2209.13172), IROS 2022. <br> |
| 102 | + |
| 103 | +
|
| 104 | + # repo: greenelab/lab-website-template |
| 105 | + tags: |
| 106 | + - Trajectory Prediction |
| 107 | + - Occupancy Prediction |
| 108 | + - Occlusion Inference |
| 109 | + - Autonomous Driving |
| 110 | + |
| 111 | + |
| 112 | + |
| 113 | +- title: Human Intention and Motion Prediction for Human-Robot Interactions |
| 114 | + # subtitle: a subtitle |
| 115 | + # group: featured |
| 116 | + image: research_images/trajectory_occupancy_prediction.png |
| 117 | + # link: https://github.com/ |
| 118 | + description: | |
| 119 | + We <br><br> |
| 120 | + |
| 121 | + **Related Publications\:** <br> |
| 122 | + 1. [CMP: Cooperative Motion Prediction with Multi-Agent Communication](https://arxiv.org/abs/2403.17916), submitted to IEEE Robotics and Automation Letters (RA-L), under review. <br> |
| 123 | + 2. [Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting](https://arxiv.org/abs/2407.09475), submitted to IEEE Robotics and Automation Letters (RA-L), under review. <br> |
| 124 | + 3. [Self-Supervised Multi-Future Occupancy Forecasting for Autonomous Driving](https://arxiv.org/abs/2407.21126), submitted to IEEE Robotics and Automation Letters (RA-L), under review. <br> |
| 125 | + 4. [Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments](https://arxiv.org/abs/2309.13893), ICRA 2024. <br> |
| 126 | + 5. [Predicting Future Spatiotemporal Occupancy Grids with Semantics for Autonomous Driving](https://arxiv.org/abs/2310.01723), IV 2024. <br> |
| 127 | + 6. [Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments](https://arxiv.org/abs/2209.13172), IROS 2022. <br> |
| 128 | + |
| 129 | +
|
| 130 | + # repo: greenelab/lab-website-template |
| 131 | + tags: |
| 132 | + - Trajectory Prediction |
| 133 | + - Occupancy Prediction |
| 134 | + - Occlusion Inference |
| 135 | + - Autonomous Driving |
| 136 | + |
86 | 137 |
|
87 | 138 |
|
88 | 139 |
|
|
0 commit comments