close-loop inference

[Hominnn]

When I am performing closed-loop reasoning, the VLM performance of my current test is very poor. It cannot complete a line and then gets stuck on this line. The program keeps generating files. How can I solve this problem?

[MeteorCollector]

Which line?

[Hominnn]

Which line?

In dev10, on a path (), the car couldn't get out after hitting a wall (probably due to a problem with the instructions output by the model). Then this path couldn't be completed. The program kept running for more than a thousand frames and couldn't proceed to the next path, so I had to force close the program.

[MeteorCollector]

This happens when the VLM is not very good... In fact, CARLA terminates some scenarios when the scenario lasts for too long, but it still waits for thousands of frames. We're also annoyed by that, so we used environment variable EARLY_STOP to shorten the time limit. You can refer to this example of start-up script to set it. However, this is not a fixed time limit, it starts to count down when the ego vehicle reaches a certain point.

To stop a scenario within certain time limit by force (which I did it before ;)), consider modifying scenario_manager_local.py.

In run_scenario function, add this at the end:

while self._running:
            self._tick_scenario()
            timestamp = GameTime.get_time()
            if timestamp > exit_point: # the stop time you set (by second)
                self._running = False
                print("[info] Time is up! Simulation stops here.") # just for debug

[Hominnn]

This happens when the VLM is not very good... In fact, CARLA terminates some scenarios when the scenario lasts for too long, but it still waits for thousands of frames. We're also annoyed by that, so we used environment variable EARLY_STOP to shorten the time limit. You can refer to this example of start-up script to set it. However, this is not a fixed time limit, it starts to count down when the ego vehicle reaches a certain point.

To stop a scenario within certain time limit by force (which I did it before ;)), consider modifying scenario_manager_local.py.

In run_scenario function, add this at the end:

while self._running:
            self._tick_scenario()
            timestamp = GameTime.get_time()
            if timestamp > exit_point: # the stop time you set (by second)
                self._running = False
                print("[info] Time is up! Simulation stops here.") # just for debug

Thanks! I'll try this method. And I find "CTRL + C" will be work. It ends the evaluation of the current route and outputs a failure, and then the evaluation of the next route.