Skip to content

Commit b49ce6c

Browse files
committed
feat: add canonical tool execution results
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
1 parent f4a7716 commit b49ce6c

176 files changed

Lines changed: 4276 additions & 1495 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/add-binding-feature/SKILL.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,9 @@ Lock these before implementing:
8787
- Does the new API need manual lifecycle and managed execute variants, or only
8888
one of them?
8989
- Does the new behavior change event fields, metadata, or scope expectations?
90+
- If tool execution is affected, does every callback, continuation, managed
91+
return, and manual end surface use the canonical `ToolExecutionResult`
92+
contract and preserve its opaque annotation?
9093
- Are docs/examples required because the intended usage changed?
9194

9295
## Key References

.agents/skills/add-middleware/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,13 @@ Refer to `docs/about-nemo-relay/concepts/middleware.mdx` for the full diagrams.
5050
specialized tool or LLM sanitizer (when applicable) -> mark or scope event
5151
sanitizer -> subscriber and exporter dispatch
5252

53+
Tool execution callbacks and each execution-intercept `next` continuation
54+
return the canonical `ToolExecutionResult { result, annotation }`. A forwarding
55+
intercept must preserve both fields in `ToolExecutionInterceptOutcome`; Relay
56+
retains `pending_marks` separately. Tool sanitize-response guardrails receive
57+
only `result`. Scope-end event sanitizers govern the annotation after Relay
58+
projects it to `category_profile.tool_result_annotation`.
59+
5360
## Core Steps
5461

5562
1. Define or reuse the callback type alias in
@@ -96,6 +103,8 @@ Follow the `add-binding-feature` skill for the cross-binding implementation chec
96103
- [ ] Callback failure policy, including fail-open behavior when required
97104
- [ ] Scope-local registration, inheritance, and cleanup on pop
98105
- [ ] Event payload semantics after middleware mutation
106+
- [ ] Tool execution result and annotation preservation, replacement, and
107+
removal when the middleware touches tool execution
99108
- [ ] Mark and scope event field semantics, including immutable identity fields
100109
- [ ] Parity coverage in every affected binding
101110

.agents/skills/contribute-docs/SKILL.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,10 @@ Use this skill for docs-only or example-heavy changes.
2828
only point at namespaced helper paths when documenting internal maintenance
2929
work
3030
- When detailed dynamic plugin guides exist, keep Rust native plugin examples,
31-
Python worker plugin examples, and `grpc-v1` protocol details on separate
31+
Python worker plugin examples, and `grpc-v2` protocol details on separate
3232
pages.
33-
- Dynamic plugin manifests in docs/examples should use
34-
`compat.relay = ">=0.5,<1.0"` unless deliberately narrower.
33+
- Dynamic plugin manifests that use native API 2 or `grpc-v2` should use
34+
`compat.relay = ">=0.8.0,<1.0"` unless deliberately narrower.
3535
- In MDX files, top-of-file comments must use JSX comment delimiters:
3636
`{/*` to open and `*/}` to close. Do not use HTML comments for MDX SPDX
3737
headers.

.agents/skills/contribute-integration/SKILL.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@ as LangChain, LangGraph, Deep Agents, or OpenClaw through its public APIs.
2828

2929
- [ ] Integration pattern follows `docs/integrate-into-frameworks/adding-scopes.mdx`
3030
- [ ] Integration uses public framework or plugin APIs
31+
- [ ] Managed tool adapters return `ToolExecutionResult` to Relay and unwrap
32+
`.result` only at the framework boundary; opaque annotations are
33+
preserved through forwarding execution intercepts
3134
- [ ] Relevant integration tests or smoke path pass
3235
- [ ] Docs updated if activation or usage changed
3336

.agents/skills/maintain-dynamic-plugins/SKILL.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Use this skill for `plugin.kind = "rust_dynamic"`, `plugin.kind = "worker"`,
2020
## Rules
2121

2222
- Keep the stable boundary explicit: native plugins cross a C ABI; worker
23-
plugins cross `grpc-v1`.
23+
plugins cross `grpc-v2`.
2424
- Do not pass Rust runtime types, trait objects, futures, or allocator-owned
2525
strings across the native dynamic-library boundary.
2626
- Typed native middleware futures run on the SDK-owned Tokio executor. Keep
@@ -30,9 +30,9 @@ Use this skill for `plugin.kind = "rust_dynamic"`, `plugin.kind = "worker"`,
3030
- Keep `relay-plugin.toml` dynamic records separate from generic runtime
3131
components. Enabled dynamic records may synthesize internal component specs;
3232
disabled records stay inspectable but unloaded.
33-
- Treat plugin Relay compatibility as normal SemVer. Typed async native plugin
34-
examples require `>=0.8.0,<1.0`; other examples use `>=0.5,<1.0` unless a
35-
plugin intentionally declares a narrower range.
33+
- Treat plugin Relay compatibility as normal SemVer. Native API 2 and `grpc-v2`
34+
examples require `>=0.8.0,<1.0` unless a plugin intentionally declares a
35+
narrower range.
3636
- Do not add tests under `src`; Rust tests belong in crate `tests/` trees and
3737
Python SDK tests belong under `python/tests`.
3838
- Native and worker plugins are trusted extensions. Document that native plugins
@@ -55,7 +55,7 @@ Use this skill for `plugin.kind = "rust_dynamic"`, `plugin.kind = "worker"`,
5555
- [ ] Top-level `doctor` reports resolved dynamic plugin and host configuration
5656
status.
5757
- [ ] When detailed dynamic plugin guides exist, they keep Rust native, Python
58-
worker, and `grpc-v1` protocol details on separate pages.
58+
worker, and `grpc-v2` protocol details on separate pages.
5959
- [ ] `justfile`, Codecov, and CI package/test workflows include new plugin
6060
crates and packages.
6161

.agents/skills/maintain-observability/SKILL.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,11 @@ OpenTelemetry projections.
5050
- [ ] Does Relay derive compliant trace and span IDs consistently across typed
5151
OpenTelemetry endpoints while preserving lifecycle parentage?
5252
- [ ] Are mark events, start/end events, and orphan cases still handled correctly?
53+
- [ ] Does a sanitized tool result annotation remain opaque under
54+
`category_profile.tool_result_annotation`, ATIF observation-result
55+
`extra.tool_result_annotation`, and the single
56+
`nemo_relay.tool.result.annotation` attribute in `full` and
57+
`openinference`, while `gen_ai` omits it?
5358
- [ ] Do examples and docs use each exporter's documented flush/deregister
5459
order before shutdown?
5560
- [ ] Are span or trajectory fields still derived from the intended event data?

crates/adaptive/tests/integration/runtime_integration_tests.rs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -618,7 +618,7 @@ async fn test_adaptive_plugin_registers_and_passes_calls_through() {
618618
.unwrap();
619619
assert_eq!(llm_result, json!({"response": "ok"}));
620620

621-
let tool_func: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args) }));
621+
let tool_func: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args.into()) }));
622622
let tool_result = tool_call_execute(
623623
nemo_relay::api::tool::ToolCallExecuteParams::builder()
624624
.name("search")
@@ -628,7 +628,7 @@ async fn test_adaptive_plugin_registers_and_passes_calls_through() {
628628
)
629629
.await
630630
.unwrap();
631-
assert_eq!(tool_result, json!({"query": "test"}));
631+
assert_eq!(tool_result.result, json!({"query": "test"}));
632632

633633
clear_plugin_configuration().unwrap();
634634
}
@@ -822,7 +822,7 @@ async fn test_top_level_plugin_registers_request_and_execution_intercepts() {
822822
.unwrap();
823823
assert_eq!(request.request.headers.get("x-plugin"), Some(&json!("set")));
824824

825-
let tool_func: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args) }));
825+
let tool_func: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args.into()) }));
826826
let tool_result = tool_call_execute(
827827
nemo_relay::api::tool::ToolCallExecuteParams::builder()
828828
.name("search")
@@ -832,7 +832,7 @@ async fn test_top_level_plugin_registers_request_and_execution_intercepts() {
832832
)
833833
.await
834834
.unwrap();
835-
assert_eq!(tool_result["x-tool-plugin"], json!(true));
835+
assert_eq!(tool_result.result["x-tool-plugin"], json!(true));
836836

837837
let llm_func: LlmExecutionNextFn =
838838
Arc::new(|_req: LlmRequest| Box::pin(async move { Ok(json!({"response": "ok"})) }));

crates/adaptive/tests/unit/intercepts_tests.rs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ async fn test_tool_intercept_calls_next() {
9999
let intercept = create_tool_execution_intercept(hot_cache);
100100

101101
let next: ToolExecutionNextFn =
102-
Arc::new(|_args| Box::pin(async move { Ok(json!({"result": "ok"})) }));
102+
Arc::new(|_args| Box::pin(async move { Ok(json!({"result": "ok"}).into()) }));
103103

104104
let result = intercept("test", json!({"input": 1}), next).await;
105105
assert!(result.is_ok());
@@ -121,7 +121,7 @@ async fn test_tool_intercept_with_populated_cache() {
121121
let intercept = create_tool_execution_intercept(hot_cache);
122122

123123
let next: ToolExecutionNextFn =
124-
Arc::new(|_args| Box::pin(async move { Ok(json!({"from_next": true})) }));
124+
Arc::new(|_args| Box::pin(async move { Ok(json!({"from_next": true}).into()) }));
125125

126126
// Should not panic and should return next's result
127127
let result = intercept("test", json!({"tool_input": "data"}), next).await;
@@ -143,7 +143,7 @@ async fn test_tool_intercept_passes_args_to_next() {
143143
let intercept = create_tool_execution_intercept(hot_cache);
144144

145145
// next captures and returns the args it received, proving pass-through
146-
let next: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args) }));
146+
let next: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args.into()) }));
147147

148148
let input = json!({"tool_arg": "value", "count": 42});
149149
let result = intercept("test", input.clone(), next).await;
@@ -317,7 +317,7 @@ async fn test_schedule_mode_intercept_waits_for_primer_before_running_follower()
317317
"follower should not call next until the primer has released the cohort"
318318
);
319319
}
320-
Ok(args)
320+
Ok(args.into())
321321
})
322322
})
323323
};

crates/adaptive/tests/unit/runtime_features_tests.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -623,7 +623,7 @@ async fn tool_parallelism_feature_registers_execution_intercept() {
623623
feature.register(&mut ctx).await.unwrap();
624624
assert_tool_execution_intercept_registered(&name);
625625

626-
let next: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args) }));
626+
let next: ToolExecutionNextFn = Arc::new(|args| Box::pin(async move { Ok(args.into()) }));
627627
let result = tool_call_execute(
628628
nemo_relay::api::tool::ToolCallExecuteParams::builder()
629629
.name("search")
@@ -633,7 +633,7 @@ async fn tool_parallelism_feature_registers_execution_intercept() {
633633
)
634634
.await
635635
.unwrap();
636-
assert_eq!(result["query"], json!("coverage"));
636+
assert_eq!(result.result["query"], json!("coverage"));
637637

638638
let mut registrations = ctx.finish();
639639
rollback_registrations(&mut registrations);

crates/cli/src/sessions/mod.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1249,7 +1249,7 @@ impl Session {
12491249
tool_call_end(
12501250
ToolCallEndParams::builder()
12511251
.handle(&handle)
1252-
.result(json!({ "status": reason }))
1252+
.execution_result(json!({ "status": reason }).into())
12531253
.metadata(json!({ "status": reason }))
12541254
.build(),
12551255
)?;
@@ -1570,7 +1570,7 @@ impl Session {
15701570
tool_call_end(
15711571
ToolCallEndParams::builder()
15721572
.handle(&handle)
1573-
.result(event.result.clone())
1573+
.execution_result(event.result.clone().into())
15741574
.metadata(merge_metadata(
15751575
event_metadata,
15761576
json!({ "status": event.status }),

0 commit comments

Comments
 (0)