Runtime stalling/freezing when parsing large JSON files concurrently

[ddroid]

Hey folks, I'm running into a weird issue where my Tokio runtime just seems to lock up entirely.

I have a web service using axum and tokio. One of the endpoints fetches a pretty massive JSON payload from an upstream service, parses it, filters some data, and saves it to a database. When this endpoint gets hit by a few concurrent users, all other endpoints on the server start timing out or responding incredibly slowly, even the basic /health check.

Here's a simplified version of what I'm doing inside the handler:

pub async fn process_data_handler() -> Result<impl IntoResponse, StatusCode> {
    let url = "https://api.example.com/massive_dataset.json";
    
    // Fetch is fine, this yields properly
    let response = reqwest::get(url).await.unwrap().text().await.unwrap();
    
    // This is where things seem to freeze
    let parsed: Vec<MyData> = serde_json::from_str(&response).unwrap();
    
    // ... filter and return
    Ok(StatusCode::OK)
}

I checked htop and my CPU is definitely spiking during the parsing phase. I thought Tokio's work-stealing scheduler would just run other tasks on the remaining threads, but it feels like the whole server grinds to a halt. I tried bumping worker_threads in the runtime builder, but it just delayed the inevitable until a few more concurrent requests came in.

Am I fundamentally misunderstanding how the scheduler handles CPU-heavy tasks?

[0x676e67]

You need to parse json and use it in the blocking thread task thread.

[ddroid]

Really Appreciate your help ❤️

[mattiapitossi]

I would suggest to read: https://docs.rs/tokio/latest/tokio/index.html#cpu-bound-tasks-and-blocking-code

[ddroid]

read, It helped a lot, Thanks for helping ❤️

[For8stt]

Yep, the misunderstanding is around what Tokio can preempt.

Tokio tasks only yield at .await points. serde_json::from_str(...) is fully synchronous CPU work, so while it’s parsing a huge payload it keeps that runtime worker busy until parsing finishes. If a few requests do that at the same time, they can occupy all worker threads and the rest of the server starts starving — including /health.

Move the CPU-heavy part off the async worker threads:

pub async fn process_data_handler() -> Result<impl IntoResponse, StatusCode> {
    let url = "https://api.example.com/massive_dataset.json";

    let response = reqwest::get(url)
        .await
        .map_err(|_| StatusCode::BAD_GATEWAY)?
        .text()
        .await
        .map_err(|_| StatusCode::BAD_GATEWAY)?;

    let parsed = tokio::task::spawn_blocking(move || {
        serde_json::from_str::<Vec<MyData>>(&response)
    })
    .await
    .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
    .map_err(|_| StatusCode::BAD_REQUEST)?;

    // filter/save/etc.
    Ok(StatusCode::OK)
}

Also add a limit, otherwise many users can still start many heavy parses:

use tokio::sync::Semaphore;
use std::sync::Arc;

static PARSE_LIMIT: std::sync::LazyLock<Arc<Semaphore>> =
    std::sync::LazyLock::new(|| Arc::new(Semaphore::new(2)));

Then acquire a permit before spawn_blocking.

So the fix is not more worker_threads, it’s:

• keep async runtime threads for I/O
• move JSON parsing/filtering to spawn_blocking
• limit how many parses can run at once

Tokio docs cover this here too:
https://docs.rs/tokio/latest/tokio/index.html#cpu-bound-tasks-and-blocking-code

[ddroid]

Thanks, it works ❤️

[ddroid]

Really Appreciate the effort you put, Love those snippets ❤️