[API Cleanup] Investigate the large input_to_model_step latency

[gtong-nv]

To reproduce:

uv run flashdreams-run-v2 crazy-robotaxi-omnidreams-fast-perf --mode native-window --backpressure-mode block -- --map apps/crazy_robotaxi/crazy_robotaxi/maps/flashdreams_raceway.robotaxi.yaml --game-mode race --race-course grand-prix --show-fps --profile-input-latency --profile-pipeline

Check the log in /tmp/crazy-robotaxi-input-trace.log

Notice the input_to_model_step_ms is between 20ms to 300ms.

[crazy-robotaxi-chunk-trace] phase=model_input_processed time_ns=3046959122714103 event_us=259197481 input_to_model_step_ms=313.935 generation=0 step=203 source=MouseUserInputEvent