Skip to content

DevOps: [profile.release] debug = 2 causes excessive memory usage and OOM failures in containerized builds #152

@jmcauley

Description

@jmcauley

Summary

The workspace Cargo.toml sets debug = 2 in [profile.release], which enables full debug symbols in all release builds. This causes significant memory pressure during linking — enough to OOM-kill the build process in containerized environments (Docker/Podman) — and produces unnecessarily large binaries for production deployments.

Observed Behavior

Running docker build for rsky-pds fails with signal: 9, SIGKILL during the final link step of rsky-pds itself (after all dependencies have compiled successfully). The process is killed by the OS out-of-memory killer, not by a compiler error. This occurs even after dedicating 8GB of RAM and 6 vCPU to the build machine.

error: could not compile `rsky-pds` (lib)

Caused by:
  process didn't exit successfully: `rustc --crate-name rsky_pds ...` (signal: 9, SIGKILL: kill)

Root Cause

Cargo.toml line 44:

[profile.release]
debug = 2

debug = 2 (full debug info) combined with opt-level = 3 (full optimization) means the compiler must process both fully-optimized IR and complete DWARF debug symbols simultaneously during codegen and linking. For a crate like rsky-pds with ~50 transitive dependencies, this creates a very high peak memory requirement at link time.

Expected Behavior

Release builds intended for production deployment should not include debug symbols by default. Debug info is valuable for profiling and debugging, but should be opt-in.

Proposed Fix

Remove debug = 2 from [profile.release] and add a dedicated profiling profile for developers who need debug symbols in an optimized build. This follows the existing named-profile pattern already present in the workspace:

[profile.release]
# debug = 2 removed — use --profile profiling for symbolized builds

[profile.profiling]
inherits = "release"
debug = 2

Developers needing symbols for a profiler (e.g., samply, perf, Instruments) can build with:

cargo build --profile profiling -p rsky-pds

All other release builds — including CI and Docker — use cargo build --release and get lean, production-appropriate binaries.

Benefits

  • Fixes OOM failures in Docker/Podman builds with reasonable memory limits
  • Reduces binary size for production deployments
  • Preserves full profiling capability via explicit opt-in
  • Consistent with the existing named-profile pattern (wasm-dev, server-dev, android-dev) already in Cargo.toml

Affected File

Cargo.toml — workspace root, affects all crates built with --release

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions