Avoid deep-copying the full request in simple_agent

[ananthsub]

Parent workstream: #3000

Slowdown path

flowchart LR
    A[Validated request] --> B[Deep-copy full model]
    B --> C[Copy input again]
    C --> D[Send growing conversation]

Loading

Current behavior

SimpleAgent._create_episode starts by calling body.model_copy(deep=True), which recursively copies the complete validated request. Each loop iteration then creates new_body with another model_copy(update=...) after concatenating body.input and new_outputs. See SimpleAgent._create_episode.

The loop mutates the local input representation and later clears and restores response usage, but the full request includes many fields that are not changed. The optimization must first audit these mutation sites so it does not expose caller-owned nested data to mutation.

Impact and provenance

Measured

The evidence report did not isolate the cost of this deep copy. It measured total per-hop work increasing from 8 ms at depth 1 to 181 ms at depth 512 as conversations re-accumulate; that result includes serialization, validation, and transport and must not be attributed to model_copy(deep=True) alone.

Expected

Copying only mutable fields should reduce allocation volume and event-loop CPU for large requests, especially when tool schemas, multimodal inputs, or token metadata are present. The size of the end-to-end improvement remains unmeasured.

Implementation constraints

• Prove that the caller's NeMoGymResponseCreateParamsNonStreaming and every caller-owned nested value remain unchanged.
• Preserve string-input normalization, tool-call accumulation, usage accumulation, trajectory collection, and per-step request semantics.
• Copy each mutable list or nested object before mutation; do not rely on undocumented Pydantic sharing behavior.
• Do not weaken model-server ingress validation.

Acceptance criteria

• The full deep copy is replaced with a documented copy strategy limited to fields the episode mutates.
• Tests prove the original request and nested input/tool structures are unchanged after success, tool calls, incomplete responses, and failures.
• Multi-step outputs, usage, cookies, trajectories, and model request bodies remain equivalent.
• Allocation and latency benchmarks show no regression for small requests.

Benchmark plan

Construct requests with small and large inputs, large tool schemas, multimodal content, and inline token metadata. Benchmark _create_episode with a stub model at 1, 8, 64, and 512 steps, measuring wall time, CPU time, peak allocations, and copied bytes. Alternate baseline and candidate runs and report the deep-copy share separately from transport and validation.

Related work/PRs

• Parent: Performance workstream: agent servers #3000
• Implementation: perf(simple-agent): avoid deep request copies #3049