Skip to content

Commit a5ad3eb

Browse files
committed
docs: clarify middleware rejection events
Signed-off-by: Alex Fournier <afournier@nvidia.com>
1 parent 3e9a097 commit a5ad3eb

1 file changed

Lines changed: 11 additions & 6 deletions

File tree

docs/about-nemo-relay/concepts/middleware.mdx

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -283,27 +283,32 @@ This ordering preserves the distinction between the families:
283283
### Rejection Path
284284

285285
A conditional-execution guardrail can reject the call during the request phase.
286-
Relay does not run request intercepts, execution intercepts, or the real
287-
callback. It sanitizes and enqueues the rejection mark, then returns the
288-
rejection to the managed caller.
286+
Relay emits a guardrail scope start/end pair for each conditional guardrail it
287+
evaluates. If a guardrail rejects the call, Relay skips the managed call start
288+
and end events and does not run request intercepts, execution intercepts, or the
289+
real callback. It then sanitizes and enqueues the rejection mark before
290+
returning the rejection to the managed caller.
289291

290292
```mermaid
291293
flowchart LR
292294
Request[Managed Request]
295+
GuardrailStart[Guardrail Scope Start]
293296
Conditional{Conditional Guardrail}
297+
GuardrailEnd[Guardrail Scope End]
294298
Mark[Sanitize and Enqueue Rejection Mark]
295299
Rejected[Return Rejection]
296300
297-
Request --> Conditional -->|rejected| Mark --> Rejected
301+
Request --> GuardrailStart --> Conditional --> GuardrailEnd
302+
GuardrailEnd -->|rejected| Mark --> Rejected
298303
```
299304

300305
### Streaming LLM Path
301306

302307
For streaming LLM flows, the same pre-execution order applies: the runtime
303-
applies `sanitize-request` guardrails and emits the LLM start event before the
308+
applies request sanitizers and emits the LLM start event before the
304309
stream execution intercept chain runs. Stream execution intercepts are the
305310
execution family for streaming provider callbacks. The runtime then collects
306-
chunks and finalizes the stream before `sanitize-response` guardrails rewrite
311+
chunks and finalizes the stream before response sanitizers rewrite
307312
the emitted end-event payload and scope-end event sanitizers run.
308313

309314
```mermaid

0 commit comments

Comments
 (0)