Skip to content

GPU OOM in C2 and WindowPoSt on v1.35.0 with FFI_USE_CUDA_SUPRASEAL=1 #13634

Description

@rvagg

Some lotus users running with FFI_USE_CUDA_SUPRASEAL=1 are seeing GPU out-of-memory panics on v1.35.0 (and later). It shows up as a Rust panic out of sppark. We've seen this both in C2 in the sealing pipeline and in generating WindowPoSt.

thread '<unnamed>' panicked at /home/admfc/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/supraseal-c2-0.1.2/src/lib.rs:151:9:
cudaGetLastError()@sppark-0.1.14/sppark/ntt/kernels/gs_mixed_radix_wide.cu:220 failed: "out of memory"
2026-02-25T23:00:55.511 INFO filecoin_proofs::api::seal > seal_commit_phase2:start: SectorId(33283)
2026-02-25T23:00:55.511 INFO filecoin_proofs::api::seal > seal_commit_phase2_circuit_proofs:start: SectorId(33283)
2026-02-25T23:00:55.511 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[68719476736]
2026-02-25T23:00:55.511 INFO filecoin_proofs::caches > found params in memory cache for STACKED[68719476736]
2026-02-25T23:00:55.528 INFO bellperson::groth16::prover::supraseal > Bellperson 0.26.0 with SupraSeal is being used!
2026-02-25T23:02:14.971 INFO bellperson::groth16::prover::supraseal > synthesis time: 79.44350428s
2026-02-25T23:02:14.971 INFO bellperson::groth16::prover::supraseal > starting proof timer

The problem seems to come from sppark, the GPU SNARK library underneath SupraSeal. We use SupraSeal via Bellperson (via the create_random_proof_batch call) for both C2 and WindowPoSt to do Groth16 when compiled with FFI_USE_CUDA_SUPRASEAL=1. When that environment variable is unset, SupraSeal doesn't get involved and neither does sppark. So the immediate workaround is to not compile with FFI_USE_CUDA_SUPRASEAL=1.

Curio users are unlikely to be impacted by this because Curio runs each of these tasks in a separate process, so any accumulation is short-lived.

The specific change looks like this commit which moved how GPU memory gets released. In practice, GPU memory isn't being reliably handed back between proofs, so a long-running lotus-miner or lotus-worker process eventually exhausts the GPU and panics. I'll take deeper technical discussion over to that repo to try and get a fix released.

For us, we can pin sppark and supraseal-c2 back to their pre-regression versions in filecoin-ffi (supraseal-c2 = 0.1.1, sppark = 0.1.11). We lose a few minor perf tweaks and there were some legitimate fixes in there too, but if none of them are biting anyone today it should be safe to wind back.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    🎉 Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions