Skip to content

memory_mb is not a hard bound: ~40% overshoot observed before the kill #30

Description

@Lutherwaves

memory_mb reads as a hard per-sandbox bound. It is not one. A process inside a sandbox reached roughly 2.8 GiB resident in a profile configured for memory_mb: 2048 before the kill landed — about 40% over.

The ceiling does hold. Pushing further got the container killed (Exited (137)), which is the correct outcome and exactly what the setting is for. The issue is only that the number where it bites is not the number in the config.

Measured

Allocating in 100 MiB steps with non-zero data (os.urandom, to defeat any zero-page deduplication), reading VmRSS from /proc/self/status inside the sandbox:

 700 MiB committed | VmRSS:  731600 kB
1400 MiB committed | VmRSS: 1448432 kB
2100 MiB committed | VmRSS: 2165260 kB   <- already past memory_mb: 2048
2800 MiB committed | VmRSS: 2882088 kB

Held for 30s to confirm it was not transient. Continuing toward 6 GiB produced the kill.

Worth noting for anyone trying to reproduce: host-side instruments do not show this. docker stats for the container and the sentry process RSS both stayed flat at tens of MiB throughout, and host MemAvailable did not move. Guest memory under this runtime is not visible to those tools the way it is for a conventional container, so the in-sandbox VmRSS is the reading to trust — and the kill is the proof the limit exists at all.

Why it matters

Anyone sizing a host multiplies memory_mb by how many sandboxes they expect to run. If the real ceiling is meaningfully higher than the configured one, that arithmetic under-provisions, and the failure mode is host memory exhaustion rather than a contained sandbox being killed.

The gap also compounds with the absence of a concurrency cap (#28): N sandboxes each able to exceed their stated bound is a larger overshoot than N times the configured figure.

What would help

Either is fine; the first is cheaper:

  1. Document it. State plainly that memory_mb is the value passed to the container runtime, that the runtime enforces it approximately rather than exactly, and that capacity planning should assume headroom above it. A worked figure ("observed ~40% over on one configuration") is more useful than a warning with no magnitude.
  2. Tighten it, if the overshoot turns out to be something the daemon can influence rather than inherent to how the runtime accounts for guest memory.

Worth establishing whether the ~1.4x ratio is stable across values before anyone treats it as a rule — it was observed at one setting, on one host, and may not generalise.

Related: the kill destroys the whole sandbox rather than the offending process, which is covered in #29.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions