VLX-Go

Vision-Language Short-Horizon Waypoint Prediction for Embodied Navigation

English | 中文

_{Overview video: VLX-Go for vision-language short-horizon waypoint prediction in embodied navigation.}

vlx_go_en.2.mp4

📺 HD version: Watch on YouTube

Overview

VLX-Go is a lightweight vision-language waypoint planner for embodied navigation. Given recent monocular frames, the current observation, and a natural-language instruction, it predicts short-horizon local waypoints that can be executed by a downstream controller in simulation or on a robot.

Rather than relying on a general-purpose VLM to describe the scene or produce text-only actions, VLX-Go maps the visual-language state directly to a compact waypoint interface. This repository focuses on target following, local navigation, dynamic obstacle avoidance, and closed-loop evaluation.

VLX-Go builds on the technical direction of OmTrackVLA and extends it toward lightweight waypoint prediction and closed-loop navigation research.

Tip

🚀 Try VLX here and explore how it enables machine to perceive, reason, and act in real time.

Community

Join the VLX community to connect with developers, explore applications, share feedback, and shape the future of multimodal AI.

Official WeChat	Discord Community

For technical support, partnerships, and community inquiries, contact us at marketing@hzlh.com.

Highlights

• 0.6B waypoint planner: A lightweight architecture for lower inference cost and easier deployment.
• Instruction-conditioned planning: Natural-language instructions specify task intent, including following, reaching, and obstacle avoidance.
• Short-horizon waypoint output: The model predicts local motion targets instead of full global routes or text-only responses.
• Temporal visual context: Recent frames help capture target motion, occlusion changes, and scene dynamics.
• Closed-loop evaluation: Navigation is evaluated through repeated observation, prediction, execution, and feedback.

Task

At each time step, VLX-Go solves a receding-horizon waypoint prediction problem.

Input

• Recent visual history: H_t = {I_{t-k}, ..., I_{t-1}}
• Current frame: I_t
• Instruction: q, for example, "follow the target person and avoid obstacles"

Output

• Short-horizon waypoint sequence: W_t = {w_1, ..., w_T}

Here, each w_i denotes a local motion target, such as position, heading, or another waypoint representation consumed by the controller. The exact dimensionality depends on the dataset and control interface.

history frames + current frame + instruction
                 |
                 v
        VLX-Go waypoint planner
                 |
                 v
       short-horizon waypoints -> controller / simulator

Method

VLX-Go separates waypoint planning from platform-specific low-level control. The planner predicts short-horizon local goals, while the downstream controller handles velocity commands, safety constraints, and dynamics.

Stage	Description
Visual encoding	Encode the current frame and recent visual history into visual features
Language conditioning	Use the instruction as the task condition for planning
Waypoint prediction	Predict short-horizon local motion targets with a 0.6B planner
Closed-loop execution	Execute predicted waypoints, collect the next observation, and predict the next segment

This rolling-horizon design is practical for dynamic scenes: targets may move, obstacles may enter the camera view, and earlier predictions can be corrected with new observations.

Training

VLX-Go is first trained from offline trajectory data and can then be refined with online simulator feedback.

Phase	Data / Signal	Objective
Offline trajectory learning	Demonstration trajectories, video frames, language instructions	Learn target-following and local waypoint generation
Online optimization	Simulator feedback, collision signals, target state, reward signals	Improve robustness to occlusion, obstacles, and closed-loop drift

Typical supervised objectives include waypoint regression, trajectory direction loss, optional velocity or action auxiliary loss, and smoothness regularization. The online stage complements supervised learning by exposing the policy to execution-time feedback that offline trajectory data may not cover.

Evaluation

VLX-Go is evaluated on the STT task of EVT-Bench.

Model	Parameters	STT SR ↑	STT TR ↑	STT CR ↓
TrackVLA	7B	85.1	78.6	1.65
NavFoM	7B	85.0	80.5	-
Qwen-RobotNav-4B	4B	77.4	90.0	6.4
Qwen-RobotNav-8B	8B	78.6	89.7	5.7
VLX-Go	0.6B	85.42	94.08	6.55

Metrics: SR is success rate, TR is tracking rate, and CR is collision rate. At the 0.6B scale, VLX-Go achieves strong success and tracking rates. Further reducing collision rate remains an important direction, especially through simulator, reward, controller, and safety-constraint tuning.

Model Checkpoint

Coming soon.

Dataset

Coming soon.