[Bugfix][Router] Run KvawareRouter /tokenize fallback in executor to avoid blocking event loop

[Asthenia0412]

Description

KvawareRouter.route_request() uses a synchronous requests.post call for the remote /tokenize fallback directly on the event loop. Under load, this blocking call stalls the event loop and can delay health checks and other concurrent requests.

The same pattern was already fixed in tokenize_prompt() by running the blocking HTTP call via asyncio.get_running_loop().run_in_executor(). This PR applies the same fix to the route_request() fallback for consistency.

Change

src/vllm_router/routers/routing_logic.py

# before
body = requests.post(remote_url, headers=headers, json=data, timeout=10).json()
token_ids = body["tokens"]

# after
loop = asyncio.get_running_loop()
response = await loop.run_in_executor(
    None,
    lambda: requests.post(remote_url, headers=headers, json=data, timeout=10),
)
token_ids = response.json()["tokens"]

Testing

• Syntax verified with ast.parse
• No behavior change; only moves the blocking call off the event loop

[gemini-code-assist]

Code Review

This pull request wraps a blocking HTTP POST request in an executor to prevent stalling the event loop in route_request. The reviewer suggests leveraging the shared aiohttp client session instead of running requests.post in an executor, which avoids thread-pool overhead and prevents socket exhaustion through connection reuse.

[gemini-code-assist]

Since route_request is an asynchronous function and has access to the request object, we can leverage the shared aiohttp client session from request.app.state.aiohttp_client_wrapper() instead of using requests.post in an executor.

Using requests.post without a session creates a new TCP connection for every fallback request, which is highly inefficient under load and can lead to socket exhaustion. Utilizing the shared aiohttp client session enables connection reuse and avoids the overhead of thread-pool execution.

            client = request.app.state.aiohttp_client_wrapper()
            async with client.post(
                remote_url, headers=headers, json=data, timeout=10
            ) as response:
                response_json = await response.json()
                token_ids = response_json["tokens"]

[Asthenia0412]

Thanks for the review! I updated the fallback to use the shared aiohttp client session from request.app.state.aiohttp_client_wrapper() instead of requests.post in an executor. This reuses TCP connections and keeps the call off the event loop. Could you take another look?