Skip to content

Proposal: portable failure passports for replaying distributed executions #4122

Description

@RitwijParmar

Problem

When a Modal function or sandbox fails after several retries, child processes, volume reads, or GPU setup steps, the useful context is spread across logs and the user's source code. Reproducing the failure usually means rebuilding the environment by hand. That makes intermittent failures in inference and batch workloads especially expensive to investigate.

The SDK already knows part of the execution boundary. It knows the image and function identity, resource request, sandbox command, mounted objects, and the calls made by the client. That information is currently lost as a single replayable object.

Proposal

Add an opt-in execution failure passport that can turn one failed invocation into a redacted, portable replay bundle.

A passport should contain:

  • the invocation and child-execution graph
  • function, image, SDK, Python, and dependency digests
  • GPU and resource configuration
  • retry and timeout decisions
  • volume and uploaded-artifact identities
  • the first failing boundary and the relevant exception chain
  • explicit side-effect boundaries
  • redacted inputs and references needed for a replay

The important part is not another log export. The bundle should say what can be replayed, what must be mocked, and what cannot be reproduced. A local inspector could then show the execution graph and generate a replay command for the reproducible part.

Suggested first slice

Keep the first version backend-neutral and opt-in:

  1. Define a versioned passport schema in the SDK.
  2. Collect the client-side execution and resource metadata without capturing secrets or raw payloads.
  3. Add deterministic redaction and stable artifact references.
  4. Provide a local inspector that validates a passport and explains its replay boundaries.
  5. Add fixtures for a retry, timeout, child-process failure, and partial upload.

The first slice does not need to rerun a remote job automatically. It should establish a stable artifact that the backend and CLI can build on later.

Why this matters

This would make a production failure useful after the original process is gone. It could also give Modal a common debugging boundary across inference endpoints, Sandboxes, and large fan-out jobs instead of adding separate diagnostics for each feature.

I have worked on incident traces, runbook retrieval, remediation gates, and audit trails across 2,500 incident scenarios. I have also built an LLM serving runtime where cold-start and latency behavior had to be measured rather than guessed. I would be interested in implementing the SDK-side schema, redaction, inspector, and replay fixtures if this direction fits Modal's roadmap.

Would the Modal team be open to discussing the schema and the smallest useful first slice before implementation?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions