Skip to content

Commit 9f422f8

Browse files
afourniernvafourniernv
authored
docs: simplify overview entry paths (#695)
#### Overview Simplify the Overview so new users see the common Relay entry paths before framework, plugin, or contributor workflows. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Limit the first-path chooser to coding-agent observation, application instrumentation, supported integrations, and observability configuration. - Move framework integration, plugin authoring, and repository development into a separate advanced section. - Replace the raw-capture procedure with brief ATOF and ATIF definitions; the existing Observability guide retains the first-export procedure. - Replace the internal registry diagram with the external application-to-runtime-to-export flow and link every shown output to its setup guide. - Validate with `just docs-linkcheck`, `just docs`, and `uv run pre-commit run --all-files`. #### Where should the reviewer start? Start with `docs/about-nemo-relay/overview.mdx`, especially "Choose Your First Path" and "How Relay Connects to Your Stack." #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Closes [RELAY-616](https://linear.app/nvidia/issue/RELAY-616/simplify-the-overview-entry-paths-and-observability-guidance) ## Summary by CodeRabbit * **Documentation** * Reframed the Relay overview around a shared runtime contract connecting applications, frameworks, observability tools, and guardrails. * Added separate integration guidance for applications and extensions, including connection boundaries and runtime-binding consistency. * Expanded guidance for exporting data through ATOF, ATIF, OpenTelemetry, and OpenInference. * Replaced the conceptual diagram with an updated flow showing applications, Relay, subscribers, exporters, and supported destinations. * Added the runtime model to the documentation table and removed the former Developer Background section. Authors: - Alex Fournier (https://github.com/afourniernv) Approvers: - https://github.com/lvojtku URL: #695
1 parent ecb04ee commit 9f422f8

1 file changed

Lines changed: 87 additions & 102 deletions

File tree

‎docs/about-nemo-relay/overview.mdx‎

Lines changed: 87 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ without rewriting the agent stack you already have. It gives coding agents,
1313
applications, framework integrations, middleware, and observability backends a
1414
shared runtime for scopes, policy, plugins, and lifecycle events.
1515

16-
Agent systems usually cross several boundaries in one request: an entrypoint
16+
Agent systems usually involve several components in one request: an entry point
1717
starts work, a model is called, tools run, subagents can branch off, and
1818
observability or policy systems need to understand what happened. Relay gives
19-
those boundaries one runtime contract instead of asking each layer to invent its
20-
own wrappers, trace vocabulary, and cleanup rules.
19+
those components one runtime contract instead of asking each layer to invent
20+
its own wrappers, trace vocabulary, and cleanup rules.
2121

2222
## Integrating With Relay
2323

@@ -27,12 +27,13 @@ run, or a framework-specific lifecycle unit.
2727

2828
<Note>
2929
Relay does not replace your agent framework, model provider, application logic,
30-
observability backend, or guardrail authoring system. It gives those systems a
31-
common runtime boundary to meet at.
30+
observability backend, or guardrail authoring system. It connects those systems
31+
through shared scopes, middleware, plugins, and lifecycle events.
3232
</Note>
3333

3434
For how Relay complements OpenTelemetry GenAI conventions and observability or
35-
evaluation products, see [How NeMo Relay Relates to Other Tooling](/about-nemo-relay/ecosystem#how-nemo-relay-relates-to-other-tooling).
35+
evaluation products, refer to
36+
[How NeMo Relay Relates to Other Tooling](/about-nemo-relay/ecosystem#how-nemo-relay-relates-to-other-tooling).
3637

3738
The first design question is simple: where can Relay observe or control the real
3839
work? The answer determines whether you should use a CLI sidecar, direct SDK
@@ -45,142 +46,126 @@ Pick the row closest to what you are trying to do.
4546
| Goal | Start With | Why |
4647
|---|---|---|
4748
| Observe Codex, Claude Code, or Hermes locally | [NeMo Relay CLI](/nemo-relay-cli/about) and [Basic Usage](/nemo-relay-cli/basic-usage) | Relay runs as a local sidecar, forwards hooks, routes provider traffic when configured, and writes observability artifacts without changing application code. |
48-
| Run the smallest binding-specific example | [Quick Start](/getting-started/quick-start) | Use this when you want a minimal Rust, Python, or Node.js workflow before adding Relay to real application code. |
49-
| Instrument application-owned LLM or tool calls | [Instrument Applications](/instrument-applications/about) | Direct SDK instrumentation gives Relay full managed-call semantics around callbacks your code owns. |
50-
| Use LangChain, LangGraph, Deep Agents, or OpenClaw | [Supported Integrations](/supported-integrations/about) | Maintained integrations use public framework or plugin APIs where they preserve enough lifecycle fidelity. |
51-
| Build a framework, host, or provider integration | [Integrate into Frameworks](/integrate-into-frameworks/about) | Integration guidance helps you choose managed wrappers, explicit lifecycle APIs, hook replay, provider codecs, or upstream support. |
52-
| Package reusable exporters, middleware, or policy | [Build Plugins](/build-plugins/about) and [Configure Plugins](/configure-plugins/about) | Plugins are the configuration-driven path for behavior that should be shared across applications or teams. |
53-
| Develop or validate the repository itself | [Development Setup](/contribute/development-setup) and [Testing and Docs](/contribute/testing-and-docs) | Use the contributor workflow when you are changing Relay source, docs, examples, bindings, or integrations. |
49+
| Instrument application-owned LLM or tool calls | [Instrument Applications](/instrument-applications/about) | Direct SDK instrumentation lets Relay run the complete lifecycle and middleware sequence around callbacks your code owns. |
50+
| Use LangChain, LangGraph, Deep Agents, or OpenClaw | [Supported Integrations](/supported-integrations/about) | Maintained integrations use public framework or plugin APIs to capture supported lifecycle events. |
51+
| Configure traces, trajectories, or raw event export | [Observability](/configure-plugins/observability/about) | Exporters consume the same lifecycle event stream and write ATOF, ATIF, OpenTelemetry, or OpenInference output. |
5452

5553
<Note>
56-
If you are unsure how much Relay you need, capture one boundary first. Confirm
57-
that Relay emits raw lifecycle events, then add normalized exports, middleware,
58-
guardrails, or adaptive behavior.
54+
To evaluate a language binding with the smallest complete example, start with
55+
[Quick Start](/getting-started/quick-start).
5956
</Note>
6057

61-
## Validate Raw Capture First
58+
Relay records canonical lifecycle events in Agent Trajectory Observability
59+
Format (ATOF). Exporters can write those events directly as
60+
[ATOF JSONL](/configure-plugins/observability/atof), project completed runs into
61+
[Agent Trajectory Interchange Format (ATIF)](/configure-plugins/observability/atif)
62+
trajectories, or translate them into typed OpenTelemetry output.
6263

63-
Start with [Agent Trajectory Observability Format (ATOF) JSONL](/configure-plugins/observability/atof),
64-
the raw canonical event stream. It shows the lifecycle events Relay actually
65-
captured before anything is translated into
66-
[Agent Trajectory Interchange Format (ATIF)](/configure-plugins/observability/atif),
67-
or a typed OpenTelemetry projection.
64+
## Build on Relay
6865

69-
A good first integration process workflow is as follows:
66+
Use these paths when you need to extend Relay instead of using an existing
67+
feature or integration.
7068

71-
1. Create or identify one scope boundary.
72-
2. Capture one LLM, tool, session, or turn boundary.
73-
3. Export ATOF JSONL and inspect the raw event stream.
74-
4. Add ATIF or a typed OpenTelemetry projection when the raw events are trustworthy.
75-
5. Add middleware only when Relay must block, sanitize, rewrite, route, or
76-
replace real execution.
69+
| Goal | Start With |
70+
|---|---|
71+
| Build a framework, host, or provider integration | [Integrate into Frameworks](/integrate-into-frameworks/about) |
72+
| Package reusable exporters, middleware, or policy | [Build Plugins](/build-plugins/about) and [Configure Plugins](/configure-plugins/about) |
73+
| Develop or validate the repository | [Development Setup](/contribute/development-setup) and [Testing and Docs](/contribute/testing-and-docs) |
74+
75+
Rust is the source of truth for runtime behavior. The Python and Node.js
76+
bindings expose the same core model for primary application use. Go and raw C
77+
FFI are experimental and source-first.
78+
79+
### Choose How Relay Connects
80+
81+
Identify where the actual LLM or tool function is invoked. If that invocation
82+
can be routed through NeMo Relay, use managed execution: NeMo Relay runs the
83+
applicable middleware and then invokes the real function. If the framework
84+
retains control but provides before-and-after notifications, translate those
85+
lifecycle hooks into NeMo Relay events. NeMo Relay can then observe the call's
86+
lifecycle but does not execute or control it. If provider-native requests and
87+
responses must be intercepted, route the real provider traffic through the
88+
NeMo Relay gateway and treat NeMo Relay as a production dependency.
89+
90+
<Warning>
91+
Do not add behavior to one primary binding without checking Rust, Python, and
92+
Node.js parity. Public behavior should stay consistent across the supported
93+
bindings.
94+
</Warning>
7795

7896
## Key Features
7997

80-
NeMo Relay offers the following features when you use it with your agent stacks:
98+
NeMo Relay offers the following features for agent applications:
8199

100+
- **Events and subscribers** so ATOF events, ATIF trajectories, and typed
101+
OpenTelemetry output come from the same runtime activity.
82102
- **Scopes** so runs, turns, tools, LLM calls, and subagents have clear
83-
ownership, parent-child lineage, cleanup boundaries, and request isolation.
103+
parent-child relationships, automatic cleanup, and request isolation.
104+
- **Marks** so point-in-time events, such as session starts, compaction, or skill
105+
loads, do not require a start and end pair.
84106
- **Managed LLM and tool calls** so the same lifecycle and middleware rules
85107
apply around each callback.
86108
- **Middleware** for the places where Relay must block, sanitize, transform,
87109
route, retry, or replace execution.
88110
- **Plugins** so reusable observability, guardrail, adaptive, and exporter
89111
behavior can be turned on from configuration.
90-
- **Events and subscribers** so raw ATOF, normalized ATIF, and typed
91-
OpenTelemetry output all come from the same runtime stream.
92112

93113
Use [Concepts](/about-nemo-relay/concepts) when you want the deeper model for
94114
scopes, events, middleware, subscribers, and plugins.
95115

96-
## Developer Background
97-
98-
Rust is the source of truth for runtime behavior. The Python and Node.js
99-
bindings expose the same core model for primary application use. Go and raw C
100-
FFI are experimental and source-first surfaces.
101-
102-
First, identify where the actual LLM or tool function is invoked. If that
103-
invocation can be routed through NeMo Relay, use managed execution: NeMo Relay
104-
runs the applicable middleware and then invokes the real function. If the
105-
framework retains control but provides before-and-after notifications, translate
106-
those lifecycle hooks into NeMo Relay events. NeMo Relay can then observe the
107-
call's lifecycle but does not execute or control it. If provider-native requests
108-
and responses must be intercepted, route the real provider traffic through the
109-
NeMo Relay gateway and treat NeMo Relay as a production dependency.
110-
111-
<Warning>
112-
Do not add behavior to one primary binding without checking Rust, Python, and
113-
Node.js parity. Public behavior should stay consistent across the supported
114-
runtime surfaces.
115-
</Warning>
116-
117116
## Documentation
118117

119118
Use the tasks below to build your understanding and set up Relay:
120119

121120
| Task | Start With |
122121
|---|---|
123122
| Install packages | [Installation](/getting-started/installation) |
124-
| Understand the mental model | [Agent Runtime Primer](/about-nemo-relay/agent-runtime-primer) |
123+
| Understand the runtime model | [Agent Runtime Primer](/about-nemo-relay/agent-runtime-primer) |
125124
| Configure plugin files | [Plugin Configuration Files](/configure-plugins/plugin-configuration-files) |
126125
| Export traces or trajectories | [Observability](/configure-plugins/observability/about) |
127126
| Tune performance with adaptive behavior | [Adaptive](/configure-plugins/adaptive/about) |
128127
| Debug trace incidents | [Trace Incident Runbook](/resources/troubleshooting/trace-incident-runbook) |
129128
| Look up symbols | [APIs](/reference/api) |
130129

131-
## Conceptual Diagram
130+
## How Relay Connects to Your Stack
132131

133-
The diagram below shows how applications, runtime components, and exporters
134-
relate to each other. Scopes define where work belongs, middleware registries
135-
define what runs around that work, and subscribers consume the lifecycle events
136-
that the core emits.
132+
The diagram shows the external flow from application work through Relay to
133+
observability output. Applications and frameworks keep ownership of their real
134+
payloads and callbacks. Relay records lifecycle events for subscribers and
135+
exporters to consume.
137136

138137
<MermaidStyles />
139138

140139
```mermaid
141-
flowchart TB
142-
Plugin[Plugin]
143-
App[Application Code / Agent Harness / Agent Framework]
144-
Framework[Framework Integration]
145-
146-
subgraph Runtime[Runtime]
147-
PluginSystem[Plugin System]
148-
Bindings[Language Bindings]
149-
Core[Rust Core Runtime]
150-
Events[Lifecycle Events]
151-
152-
subgraph RuntimeState[Runtime State]
153-
Registry[<strong>Middleware Registries</strong><br/>what runs around work]
154-
Scope[<strong>Scope Stack</strong><br/>where work belongs]
155-
end
156-
157-
Subs[Subscribers / Exporters]
158-
159-
PluginSystem --->|installs| Registry
160-
PluginSystem ----->|installs| Subs
161-
Bindings --> Core
162-
Core -->|emits| Events -->|consumed by| Subs
163-
Core -->|updates| Scope
164-
Core -->|resolves| Registry
140+
flowchart LR
141+
Sources[Application Code / Agent Harness / Agent Framework]
142+
Runtime[Relay Runtime]
143+
Consumers[Subscribers / Exporters]
144+
145+
subgraph Outputs[Outputs and Destinations]
146+
ATOF[ATOF JSONL]
147+
ATIF[ATIF Trajectories]
148+
OTEL[OpenTelemetry]
149+
OI[OpenInference]
165150
end
166151
167-
App -->|registers| Plugin
168-
App -->|uses| Framework
169-
App -->|configures/initializes| PluginSystem
170-
App -->|uses| Bindings
171-
Framework -->|calls| Bindings
172-
Plugin -->|registers with| PluginSystem
173-
174-
class Runtime grey-lightest;
175-
class RuntimeState grey-lightest;
176-
class App purple-lightest;
177-
class Framework yellow-lightest;
178-
class Plugin blue-lightest;
179-
class Bindings green-lightest;
180-
class PluginSystem green-light;
181-
class Core green-light;
182-
class Scope green-light;
183-
class Registry green-light;
184-
class Events green-light;
185-
class Subs green-light;
152+
Sources -->|managed calls, hooks, or routed traffic| Runtime
153+
Runtime -->|canonical lifecycle events| Consumers
154+
Consumers --> ATOF
155+
Consumers --> ATIF
156+
Consumers --> OTEL
157+
Consumers --> OI
158+
159+
class Sources purple-lightest;
160+
class Runtime green-light;
161+
class Consumers green-lightest;
162+
class Outputs grey-lightest;
163+
class ATOF,ATIF,OTEL,OI blue-lightest;
186164
```
165+
166+
Configure the output that matches your destination:
167+
168+
- [ATOF JSONL](/configure-plugins/observability/atof) for the canonical event stream.
169+
- [ATIF trajectories](/configure-plugins/observability/atif) for replay, analysis, and evaluation.
170+
- [OpenTelemetry](/configure-plugins/observability/opentelemetry) for OTLP-compatible backends.
171+
- [OpenInference](/configure-plugins/observability/openinference) for OpenInference-compatible tracing.

0 commit comments

Comments
 (0)