Skip to content

fix(code-gen): replace Manager proxies with bounded IPC - #3058

Open
ananthsub wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
ananthsub:ananthsub/bound-codegen-process-ipc
Open

fix(code-gen): replace Manager proxies with bounded IPC#3058
ananthsub wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
ananthsub:ananthsub/bound-codegen-process-ipc

Conversation

@ananthsub

Copy link
Copy Markdown
Contributor

Summary

  • replace each verification's multiprocessing.Manager helper and two ListProxy objects with one versioned payload over a one-way pipe
  • pass already validated unit-test dictionaries across Ray instead of dumping to JSON and parsing them again
  • cap the input-scaled child backstop at a configurable 600 seconds and cap accepted result payloads at 16 MiB
  • preserve one fresh isolated child per submission and request non-forceful Ray cancellation when the caller disconnects

Correctness and reliability

Timeout, child crash, EOF, malformed or oversized payload, invalid tests, and interruption fail closed with the existing failed-test shape. Every path closes both pipe endpoints and kills/joins a live child. The worker remains single-use because LiveCodeBench's reliability guard mutates process-global state.

Validation

  • uv run --extra dev pytest resources_servers/code_gen/tests -q — 73 passed
  • scoped pre-commit — passed
  • real child tests verify call-based/plain-dict and legacy JSON input parity
  • 16 forced child crashes leave no descendants or additional file descriptors

Focused IPC benchmark

For a ten-result payload over 50 local runs:

  • Manager helper + ListProxy: 7.464 ms median, 8.758 ms p95, two child processes per verification
  • one-way Pipe: 1.167 ms median, 1.636 ms p95, one child process per verification

This isolates process/IPC overhead; it is not an end-to-end rollout throughput claim.

Closes #3028

Remove the per-verification Manager helper while bounding child lifetime and result payloads without weakening process isolation or cleanup.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace code_gen per-verify Manager processes with bounded IPC

1 participant