Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion docs/dsl.md
Original file line number Diff line number Diff line change
Expand Up @@ -741,7 +741,9 @@ Tool("get_data", "Get user data", func() {

### Display Hint Templates

`CallHintTemplate` and `ResultHintTemplate` configure Go templates for UI display:
`CallHintTemplate` and `ResultHintTemplate` configure Go templates for UI display.
These templates are rendered by the runtime against the tool's **typed** payload/result structs
and surfaced via hook + stream events as `DisplayHint` (call) and result previews (result).

```go
Tool("search", "Search documents", func() {
Expand All @@ -761,6 +763,21 @@ Tool("search", "Search documents", func() {
Templates are compiled with `missingkey=error`. Keep hints concise (≤140 characters recommended).
Template variables use Go field names (e.g., `.Query`, `.Limit`), not JSON keys.

**Runtime contract:**

- Tool call scheduled events default to `DisplayHint==""` at construction time. The runtime may enrich
and persist a **durable default** hint when it can decode the typed payload and execute the template.
- If you set `DisplayHint` explicitly (non-empty) before publishing the hook event, the runtime treats it
as authoritative and will not overwrite it.
- If typed decoding fails, the runtime leaves `DisplayHint` empty (strict contract: no rendering against
raw JSON bytes).

**Per-consumer overrides (optional):**

If you need a different hint for a specific deployment/consumer (e.g., UI wording), configure a runtime
override via `runtime.WithHintOverrides`. Overrides take precedence over DSL templates for streamed
`tool_start` events.

### BoundedResult

`BoundedResult` marks a tool's result as a bounded view over potentially larger data. When set:
Expand Down
4 changes: 2 additions & 2 deletions docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,8 +443,8 @@ policies, and MCP servers within Goa service designs.
| `Tags(...)` | Attach metadata labels for filtering/categorization |
| `BindTo(method)` or `BindTo(service, method)` | Bind tool to service method implementation |
| `Inject(fields...)` | Mark fields as infrastructure-only (hidden from LLM) |
| `CallHintTemplate(tmpl)` | Go template for call display hint |
| `ResultHintTemplate(tmpl)` | Go template for result display hint |
| `CallHintTemplate(tmpl)` | Go template for tool call `DisplayHint` (typed payload; rendered by runtime) |
| `ResultHintTemplate(tmpl)` | Go template for tool result display (typed result; rendered by runtime) |
| `BoundedResult()` | Mark result as bounded view over larger data |
| `ResultReminder(text)` | Static system reminder injected after tool result |

Expand Down
22 changes: 20 additions & 2 deletions docs/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -492,14 +492,32 @@ type ToolsetRegistration struct {
Specs []tools.ToolSpec // JSON codecs and schemas
TaskQueue string // Optional queue override
Inline bool // Execute in workflow context
CallHints map[tools.Ident]*template.Template // Display hint templates
ResultHints map[tools.Ident]*template.Template // Result preview templates
CallHints map[tools.Ident]*template.Template // Tool call DisplayHint templates (typed payload only)
ResultHints map[tools.Ident]*template.Template // Tool result preview templates (typed result only)
PayloadAdapter func(...) // Pre-decode transformation
ResultAdapter func(...) // Post-encode transformation
AgentTool *AgentToolConfig // Agent-as-tool configuration
}
```

### Tool Call Display Hints (DisplayHint)

The runtime can surface user-facing hints for tool calls (for example in UIs) via the `DisplayHint` field on
hook + stream events.

Contract:

- Hook constructors do not render hints. Tool call scheduled events default to `DisplayHint==""`.
- The runtime may enrich and persist a **durable default** hint at publish time by decoding the typed tool
payload using generated codecs and executing the `CallHintTemplate` (if registered).
- When typed decoding fails or no template is registered, the runtime leaves `DisplayHint` empty. Hints are
never rendered against raw JSON bytes.
- If a producer explicitly sets `DisplayHint` (non-empty) before publishing the hook event, the runtime treats
it as authoritative and does not overwrite it.

For per-consumer wording changes, configure `runtime.WithHintOverrides` on the runtime. Overrides take precedence
over DSL-authored templates for streamed `tool_start` events.

### Tool Implementation Patterns

**Method-backed tools** — Generated from `BindTo` DSL:
Expand Down
12 changes: 4 additions & 8 deletions runtime/agent/hooks/events.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ import (
"goa.design/goa-ai/runtime/agent/prompt"
"goa.design/goa-ai/runtime/agent/rawjson"
"goa.design/goa-ai/runtime/agent/run"
rthints "goa.design/goa-ai/runtime/agent/runtime/hints"
"goa.design/goa-ai/runtime/agent/telemetry"
"goa.design/goa-ai/runtime/agent/toolerrors"
"goa.design/goa-ai/runtime/agent/tools"
Expand Down Expand Up @@ -769,12 +768,6 @@ func (e *AwaitExternalToolsEvent) Type() EventType { return AwaitExternalTools }
// canonical JSON arguments for the scheduled tool; queue is the activity queue name.
// ParentToolCallID and expectedChildren are optional (empty/0 for top-level calls).
func NewToolCallScheduledEvent(runID string, agentID agent.Ident, sessionID string, toolName tools.Ident, toolCallID string, payload rawjson.RawJSON, queue string, parentToolCallID string, expectedChildren int) *ToolCallScheduledEvent {
// Compute a best-effort call hint once at emit time so all subscribers can
// reuse it. The payload is the canonical JSON arguments; templates that
// depend on typed structs will be rerun by higher-level decorators (e.g.,
// the runtime hinting sink) when needed.
displayHint := rthints.FormatCallHint(toolName, payload.RawMessage())

be := newBaseEvent(runID, agentID)
be.sessionID = sessionID
return &ToolCallScheduledEvent{
Expand All @@ -785,7 +778,10 @@ func NewToolCallScheduledEvent(runID string, agentID agent.Ident, sessionID stri
Queue: queue,
ParentToolCallID: parentToolCallID,
ExpectedChildrenTotal: expectedChildren,
DisplayHint: displayHint,
// DisplayHint is computed by the runtime at publish time using typed payloads
// and registered templates. This keeps the contract strict: hints are never
// rendered against raw JSON bytes.
DisplayHint: "",
}
}

Expand Down
41 changes: 40 additions & 1 deletion runtime/agent/runtime/hook_activity.go
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
package runtime

import (
"bytes"
"context"
"encoding/json"
"errors"
"time"

"goa.design/goa-ai/runtime/agent/hooks"
"goa.design/goa-ai/runtime/agent/prompt"
"goa.design/goa-ai/runtime/agent/runlog"
rthints "goa.design/goa-ai/runtime/agent/runtime/hints"
"goa.design/goa-ai/runtime/agent/session"
)

Expand Down Expand Up @@ -36,6 +39,15 @@ func (r *Runtime) hookActivity(ctx context.Context, input *HookActivityInput) er
if err != nil {
return err
}
payload := append([]byte(nil), input.Payload...)
if e, ok := evt.(*hooks.ToolCallScheduledEvent); ok {
if enriched := r.enrichToolCallScheduledHint(ctx, e); enriched {
reencoded, err := hooks.EncodeToHookInput(e, input.TurnID)
if err == nil {
payload = append([]byte(nil), reencoded.Payload.RawMessage()...)
}
}
}
// Tool call argument deltas are best-effort UX signals. They are intentionally
// excluded from the canonical run event log to avoid bloating durable history.
//
Expand All @@ -48,7 +60,7 @@ func (r *Runtime) hookActivity(ctx context.Context, input *HookActivityInput) er
SessionID: input.SessionID,
TurnID: input.TurnID,
Type: input.Type,
Payload: append([]byte(nil), input.Payload...),
Payload: payload,
Timestamp: time.UnixMilli(evt.Timestamp()).UTC(),
}); err != nil {
return err
Expand Down Expand Up @@ -81,6 +93,33 @@ func (r *Runtime) hookActivity(ctx context.Context, input *HookActivityInput) er
return nil
}

func (r *Runtime) enrichToolCallScheduledHint(ctx context.Context, evt *hooks.ToolCallScheduledEvent) bool {
if evt == nil {
return false
}
if evt.DisplayHint != "" {
return false
}
raw := normalizeHintPayloadJSON(evt.Payload.RawMessage())
typed, err := r.unmarshalToolValue(ctx, evt.ToolName, raw, true)
if err != nil || typed == nil {
return false
}
if hint := rthints.FormatCallHint(evt.ToolName, typed); hint != "" {
evt.DisplayHint = hint
return true
}
return false
}

func normalizeHintPayloadJSON(raw json.RawMessage) json.RawMessage {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) == 0 || bytes.Equal(trimmed, []byte("null")) {
return json.RawMessage("{}")
}
return raw
}

func (r *Runtime) updateRunMetaFromHookEvent(ctx context.Context, evt hooks.Event) error {
if evt == nil {
return errors.New("runtime: hook event is nil")
Expand Down
60 changes: 60 additions & 0 deletions runtime/agent/runtime/hook_activity_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,13 @@ import (
"github.com/stretchr/testify/require"
"goa.design/goa-ai/runtime/agent/hooks"
"goa.design/goa-ai/runtime/agent/prompt"
"goa.design/goa-ai/runtime/agent/rawjson"
"goa.design/goa-ai/runtime/agent/runlog"
rthints "goa.design/goa-ai/runtime/agent/runtime/hints"
"goa.design/goa-ai/runtime/agent/session"
sessioninmem "goa.design/goa-ai/runtime/agent/session/inmem"
"goa.design/goa-ai/runtime/agent/telemetry"
"goa.design/goa-ai/runtime/agent/tools"
)

type recordingRunlog struct {
Expand Down Expand Up @@ -126,6 +130,62 @@ func TestHookActivityAppendFailureAbortsPublish(t *testing.T) {
require.Nil(t, published)
}

func TestHookActivity_EnrichesToolCallScheduledDisplayHintInRunlog(t *testing.T) {
t.Parallel()

toolID := tools.Ident("runtime.hints.test.scheduled")
rthints.RegisterCallHint(toolID, mustTemplate(t, toolID, "Checking {{.Resolution}} energy rates"))

rl := &recordingRunlog{}
bus := hooks.NewBus()
store := sessioninmem.New()

rt := &Runtime{
RunEventStore: rl,
Bus: bus,
SessionStore: store,
logger: telemetry.NoopLogger{},
toolSpecs: map[tools.Ident]tools.ToolSpec{
toolID: newTypedHintSpec(toolID),
},
}

now := time.Now().UTC()
_, err := store.CreateSession(context.Background(), "sess-1", now)
require.NoError(t, err)
require.NoError(t, store.UpsertRun(context.Background(), session.RunMeta{
AgentID: "svc.agent",
RunID: "run-1",
SessionID: "sess-1",
Status: session.RunStatusPending,
StartedAt: now,
UpdatedAt: now,
}))

ev := hooks.NewToolCallScheduledEvent(
"run-1",
"svc.agent",
"sess-1",
toolID,
"call-1",
rawjson.RawJSON([]byte(`{"resolution":"hourly"}`)),
"queue",
"",
0,
)
// Hooks constructors do not render hints. The runtime fills in a durable default
// hint (when possible) using typed payloads at publish time.
require.Empty(t, ev.DisplayHint)

input, err := hooks.EncodeToHookInput(ev, "turn-1")
require.NoError(t, err)

require.NoError(t, rt.hookActivity(context.Background(), input))
require.Len(t, rl.events, 1)
require.Equal(t, hooks.ToolCallScheduled, rl.events[0].Type)
require.Contains(t, string(rl.events[0].Payload), "Checking hourly energy rates")
}

func TestHookActivityAccumulatesPromptRefsOnRunMeta(t *testing.T) {
t.Parallel()

Expand Down
26 changes: 26 additions & 0 deletions runtime/agent/runtime/runtime.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,17 @@ import (
)

type (
// HintOverrideFunc can override the call hint for a tool invocation.
//
// Contract:
// - Returning (hint, true) selects hint as the DisplayHint, even when a DSL
// template exists.
// - Returning ("", false) indicates no override applies and the runtime should
// use its default behavior.
// - The payload value is the typed payload decoded via the tool payload codec
// when possible; it may be nil when decoding fails.
HintOverrideFunc func(ctx context.Context, tool tools.Ident, payload any) (hint string, ok bool)

// Runtime orchestrates agent workflows, policy enforcement, memory persistence,
// and event streaming. It serves as the central registry for agents, toolsets,
// and models. All public methods are thread-safe and can be called concurrently.
Expand Down Expand Up @@ -150,6 +161,8 @@ type (
// It is used to require explicit operator approval before executing certain tools.
// See ToolConfirmationConfig for details.
toolConfirmation *ToolConfirmationConfig

hintOverrides map[tools.Ident]HintOverrideFunc
}

// Options configures the Runtime instance. All fields are optional except Engine
Expand Down Expand Up @@ -194,6 +207,10 @@ type (
// tools (for example, requiring explicit operator approval before executing
// additional tools that are not marked with design-time Confirmation).
ToolConfirmation *ToolConfirmationConfig

// HintOverrides optionally overrides DSL-authored call hints for specific tools
// when streaming tool_start events.
HintOverrides map[tools.Ident]HintOverrideFunc
}

// RuntimeOption configures the runtime via functional options passed to NewWith.
Expand Down Expand Up @@ -662,6 +679,7 @@ func newFromOptions(opts Options) *Runtime {
workers: opts.Workers,
reminders: reminder.NewEngine(),
toolConfirmation: opts.ToolConfirmation,
hintOverrides: opts.HintOverrides,
}
rt.PromptRegistry.SetObserver(rt.onPromptRendered)
// Install runtime-owned toolsets before any agent registration so planners
Expand Down Expand Up @@ -906,6 +924,14 @@ func WithToolConfirmation(cfg *ToolConfirmationConfig) RuntimeOption {
return func(o *Options) { o.ToolConfirmation = cfg }
}

// WithHintOverrides configures per-tool call hint overrides.
//
// When provided, overrides take precedence over DSL-authored CallHint templates
// when streaming tool_start events. Only tools present in the map are considered.
func WithHintOverrides(m map[tools.Ident]HintOverrideFunc) RuntimeOption {
return func(o *Options) { o.HintOverrides = m }
}

// WithWorker configures the worker for a specific agent. Engines that support
// worker polling use this configuration to bind the agent to a specific queue.
// If unspecified, a default worker configuration is used.
Expand Down
33 changes: 24 additions & 9 deletions runtime/agent/runtime/runtime_hints_sink.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"encoding/json"

"goa.design/goa-ai/runtime/agent/rawjson"
rthints "goa.design/goa-ai/runtime/agent/runtime/hints"
"goa.design/goa-ai/runtime/agent/stream"
"goa.design/goa-ai/runtime/agent/tools"
Expand Down Expand Up @@ -32,10 +33,20 @@ func (h *hintingSink) Send(ctx context.Context, ev stream.Event) error {
switch e := ev.(type) {
case stream.ToolStart:
data := e.Data
if data.DisplayHint == "" {
if typed := h.decodePayload(ctx, tools.Ident(data.ToolName), data.Payload); typed != nil {
if s := rthints.FormatCallHint(tools.Ident(data.ToolName), typed); s != "" {
data.DisplayHint = s

toolName := tools.Ident(data.ToolName)
override := h.rt.hintOverrides[toolName]
if data.DisplayHint == "" || override != nil {
if typed := h.decodePayload(ctx, toolName, data.Payload); typed != nil {
if override != nil {
if hint, ok := override(ctx, toolName, typed); ok {
data.DisplayHint = hint
}
}
if data.DisplayHint == "" {
if s := rthints.FormatCallHint(toolName, typed); s != "" {
data.DisplayHint = s
}
}
}
}
Expand All @@ -58,16 +69,20 @@ func (h *hintingSink) Close(ctx context.Context) error {
// runtime's tool codecs.
//
// Contract:
// - Tool payloads are canonical JSON values for the tool payload schema.
// - A missing/empty payload is normalized to "{}" (empty object) so tools with
// empty payload schemas still render call hints deterministically.
// - Hints are only rendered from typed payloads produced by registered codecs.
// If decode fails, this function returns nil.
// - Tool payloads are canonical JSON values for the tool payload schema.
// - A missing/empty payload is normalized to "{}" (empty object) so tools with
// empty payload schemas still render call hints deterministically.
// - Hints are only rendered from typed payloads produced by registered codecs.
// If decode fails, this function returns nil.
func (h *hintingSink) decodePayload(ctx context.Context, tool tools.Ident, payload any) any {
raw := json.RawMessage("{}")
switch v := payload.(type) {
case nil:
// Keep canonical empty object.
case rawjson.RawJSON:
if len(v) > 0 {
raw = v.RawMessage()
}
case json.RawMessage:
if len(v) > 0 {
raw = v
Expand Down
Loading