You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: index.html
+11-11Lines changed: 11 additions & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -358,7 +358,7 @@ <h2>Abstract</h2>
358
358
<p>Vision-Language Models (VLMs) are increasingly proposed for autonomous driving tasks, yet their performance on sequential driving scenes remains poorly characterized, particularly regarding how input configurations affect their capabilities.</p>
359
359
<p>We introduce <strong>VENUSS</strong>, a framework for systematic sensitivity analysis of VLM performance on sequential driving scenes, establishing baselines for future research. Building upon existing datasets, VENUSS extracts temporal sequences from driving videos and generates structured evaluations across custom categories.</p>
360
360
<p>By comparing 25+ existing VLMs across 2,600+ scenarios, we reveal how even top models achieve only <strong>57% accuracy</strong>, not matching human performance in similar constraints (65%) and exposing significant capability gaps. Our analysis shows that VLMs excel with static object detection but struggle with understanding the vehicle dynamics and temporal relations.</p>
361
-
<p>VENUSS offers the first systematic sensitivity analysis of VLMs focused on how input image configurations — resolution, frame count, temporal intervals, spatial layouts, and presentation modes — affect performance on sequential driving scenes.</p>
361
+
<p>VENUSS offers the first systematic sensitivity analysis of VLMs focused on how input image configurations – resolution, frame count, temporal intervals, spatial layouts, and presentation modes – affect performance on sequential driving scenes.</p>
<p>VENUSS is designed to be dataset-agnostic. Categories are automatically extracted from each dataset's textual descriptions. We release VENUSS with configurations for four datasets:</p>
396
396
<ul>
397
-
<li><strong>CoVLA</strong>: 7 <em>behavioral</em> categories (motion, direction, speed, following, acceleration, traffic lights, curvature) — extracted from natural language captions.</li>
0 commit comments