Skip to content

Commit 85a3c44

Browse files
committed
✨ feat(graph,api,console): probe agents emit structured result cards β€” agent-aware scg_results projection + per-result confidence (#102) (#103)
Implements #102 β€” search probes now return their findings as structured result cards that render on the search page, with the richest per-item detail the connector yields (title, source, snippet, url, kind, relevance, **confidence**). ## New features - **Probe-emitted result cards.** Each `scg-path-probe` emits the hits its pathway grounded through the existing `scg_results` validate-only tool (already bound to probes via the capability gate β€” **no new transport, no new tool**), once, right before its evidence block. The root's step 5.5 narrows to inline-grounded hits so probes own their cards (playbook-level dedup). Probe emits project as `result` SSE events + `RunPayload.results` with probe-salted stable ids `r-<run8>-<agent8>-<n>` β€” collision-free across concurrent emitters, deterministic live↔settle (same transcript event β‡’ same id β‡’ dedup holds). - **Per-result `confidence` on the wire** (additive): `SearchResult.confidence` carries the emitting agent's per-card certainty verbatim (BE `schemas.py` + FE `agenticSearch.ts`); the existing relevance fold is preserved. `ResultCard` renders a mono `%` chip beside the relevance dot only when present β€” honest-or-absent, never an invented 0%. ## Stability β€” corrected root cause The #95 premise "*probes run in their own sessions; their tool_results never appear here*" is **false**: a child loop inherits the parent's `event_logger` (`mewbo_core/agent_context.py:85`) and every `tool_result` payload is stamped with its `agent_id` (`tool_use_loop.py:2305`). Verified live in Mongo β€” e.g. session `eb132add…` (run-35a4f94e39) carries **89 probe tool_results** in the root transcript; 8/8 sampled search runs agree. Consequences fixed: - **Coordinator-lane mislabeling (live bug):** probe tool calls were being digested into the coordinator lane as root activity. Live streamer (`_probe_emitter`/`_project_probe_tool`) and settle (`probe_ids` threaded through `_build_coordinator_lines`/`_assign_lane_slots`/`_build_results`) now classify every `tool_result` by `agent_id ∈ probe lanes` β€” same classifier both sides, so live and settle agree. - **Id collisions:** a probe emit would previously mint the same `r-<run8>-<n>` ids as the root's and be silently dropped by the dedup. **Deliberate non-goal:** `scg_results` stays **non-terminal** for probes β€” the probe's terminal is its `EVIDENCE`/`NO DATA` stop summary (the #86 seam feeding per-lane evidence + synthesis metrics); a terminal emit would erase it. ## Validation (observed locally) - `pytest tests` β€” **3561 passed** (incl. 4 new cases: probe live attribution + settle parity, coordinator-pollution regression, mixed-emit id-collision regression, confidence wire semantics). - `ruff check .` β€” All checks passed; `make typecheck` β€” Success: no issues found in 230 source files. - Console: `npx tsc --noEmit` exit 0, eslint exit 0, `vitest run src/components/agentic_search` β€” 24 passed (incl. the new confidence-chip test). - **Caveat:** validated via unit/contract tests over real-shaped transcripts (mirroring live Mongo event shapes); a live end-to-end LLM search run was not exercised. Probe emission also depends on the model following the updated playbook β€” the projection handles zero-emit probes gracefully (no behavior change for them). ## Docs `agentic_search/scg/CLAUDE.md` + `plugins/scg/CLAUDE.md` correct the false own-sessions premise and record the classification + id-salting invariants; console `agentic_search/CLAUDE.md` notes the per-card confidence stance. Closes #102 Reviewed-on: https://git.hurricane.home/bearlike/Assistant/pulls/103 Co-authored-by: Krishnakanth Alagiri <mail@kanth.tech> Co-committed-by: Krishnakanth Alagiri <mail@kanth.tech>
1 parent 96ae473 commit 85a3c44

13 files changed

Lines changed: 428 additions & 83 deletions

File tree

β€Žapps/mewbo_api/src/mewbo_api/agentic_search/scg/CLAUDE.mdβ€Ž

Lines changed: 28 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -106,18 +106,34 @@ keyed everything off `sub_agent` events β†’ blank trace, `results=[]`,
106106
never its entries. The lane has no `stop` lifecycle β€” its `agent_done`
107107
fires at settle, `empty = no data-bearing probe AND no results`. Slots come
108108
from merged first-seen order (`_assign_lane_slots`) so settle reproduces
109-
the live interleaving. Probe `tool_result`s never appear here β€” probes run
110-
in their own sessions; this session's bus only carries the root's.
111-
- **`scg_results` = transcript-as-transport.** The tool
112-
(`mewbo_graph.plugins.scg.results`, granted via `TRAVERSAL_TOOLS`)
113-
validates (≀50 entries, `extra="forbid"`, relevance/confidence 0..1) and
114-
returns `{ok, count}` β€” it writes NOTHING (layering: the library never
115-
touches the api run store; no MapPhaseSink-style DI needed because the
116-
transcript already reaches the api). `ResultsProjection` maps entries β†’
117-
`SearchResult` with stable ids `r-<run_id8>-<n>` β€” that id is the
118-
live↔settle dedup key. Entry `confidence` folds into `relevance` only when
119-
`relevance` is absent (no wire field for it). The playbook's terminal
120-
discipline: the ROOT emits once, before synthesis; probes never call it.
109+
the live interleaving.
110+
- **Probe tool_results ARE on this transcript/bus (#102 β€” the #95
111+
"own-sessions" premise was WRONG, verified live in Mongo).** A child loop
112+
inherits the parent's `event_logger` (core `AgentContext.child`), and every
113+
`tool_result` payload carries the emitting `agent_id` β€” so EVERY
114+
`tool_result` must be classified `agent_id ∈ probe lanes` before
115+
projection (live `_probe_emitter` / settle `probe_ids` from the trace; a
116+
spawn's `sub_agent` `start` always precedes the child's first tool call, so
117+
the lane is known in time). Unclassified, probe tool calls mislabel into
118+
the coordinator lane. A probe's `scg_results` projects as ITS result
119+
cards; its other tool calls project NOWHERE (the probe lane stays
120+
lifecycle-only β€” the #86 evidence rides the stop summary).
121+
- **`scg_results` = transcript-as-transport, EVERY search agent emits.** The
122+
tool (`mewbo_graph.plugins.scg.results`, granted via `TRAVERSAL_TOOLS` +
123+
bound to probes by the capability gate) validates (≀50 entries,
124+
`extra="forbid"`, relevance/confidence 0..1) and returns `{ok, count}` β€”
125+
it writes NOTHING (layering: the library never touches the api run store;
126+
no MapPhaseSink-style DI needed because the transcript already reaches the
127+
api). `ResultsProjection` maps entries β†’ `SearchResult` with stable ids:
128+
`r-<run_id8>-<n>` (root) / `r-<run_id8>-<agent8>-<n>` (probe emit, salted
129+
so concurrent emitters never collide) β€” that id is the live↔settle dedup
130+
key. Entry `confidence` rides the wire verbatim (`SearchResult.confidence`,
131+
#102) AND folds into `relevance` only when `relevance` is absent. Playbook
132+
discipline: each probe emits ONCE before its evidence block; the ROOT
133+
emits once, before synthesis, ONLY for hits it grounded inline (never
134+
re-emitting a probe's β€” duplicate cards have distinct ids, the playbook is
135+
the dedup). Emitting is NOT terminal for anyone β€” a probe's terminal stays
136+
the stop-summary evidence block (#86).
121137
- **Metrics provenance** (`_synthesis_metrics`): `sources_count` = distinct
122138
grounding sources = data-bearing probe lanes (keyed by `agent_id` β€” the
123139
wire `source_id` is the shared parent grouping key, useless for

β€Žapps/mewbo_api/src/mewbo_api/agentic_search/scg/orchestrated_runner.pyβ€Ž

Lines changed: 45 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -306,18 +306,25 @@ def _settle(
306306
return None
307307

308308
trace = self._build_trace(records)
309+
# Probe lanes are the classification key for tool_result events (#102):
310+
# a child loop inherits the parent's event_logger, so probe tool calls
311+
# land on THIS transcript stamped with the probe's agent_id β€” they must
312+
# never read as root/coordinator activity. Same classifier as the live
313+
# streamer (agent_id ∈ probe lanes), so live and settle agree.
314+
probe_ids = {agent.agent_id for agent in trace}
309315
# The root coordinator's own tool activity (#95): a root-inline run (fast
310316
# tier, no probe sub-agents) streams nothing through the probe lanes, so
311317
# the root's ``tool_result`` events are projected as one extra lane. Its
312318
# slot is its first-seen ordinal across the MERGED stream (probes +
313319
# coordinator) so the settle reconcile honours transcript order exactly as
314320
# the live path does (coordinator opens when its first tool_result lands).
315321
# This re-slots the probe lanes in place to leave room for it.
316-
coordinator_slot = self._assign_lane_slots(records, trace)
317-
coordinator_lines = self._build_coordinator_lines(records)
322+
coordinator_slot = self._assign_lane_slots(records, trace, probe_ids)
323+
coordinator_lines = self._build_coordinator_lines(records, probe_ids)
318324
# ``scg_results`` emits the discrete result cards (transcript-as-transport):
319-
# the root calls it once before synthesis; the api projects its entries.
320-
results = self._build_results(run.run_id, records)
325+
# the root emits once before synthesis; each probe may emit its own cards
326+
# (#102). The api projects every emit, ids salted by the emitting probe.
327+
results = self._build_results(run.run_id, records, probe_ids)
321328
# A run is data-bearing iff any probe returned data OR any result emitted.
322329
with_data_probes = [a for a in trace if not ProbeTrace.is_dead_end(a.result)]
323330
has_data = bool(with_data_probes) or bool(results)
@@ -489,15 +496,18 @@ def _build_trace(records: list[dict[str, Any]]) -> list[TraceAgent]:
489496

490497
@staticmethod
491498
def _assign_lane_slots(
492-
records: list[dict[str, Any]], trace: list[TraceAgent]
499+
records: list[dict[str, Any]],
500+
trace: list[TraceAgent],
501+
probe_ids: set[str],
493502
) -> int | None:
494503
"""Re-slot probe lanes by MERGED first-seen order; return coordinator slot.
495504
496505
The live streamer assigns a lane's slot from ``len(self._order)`` at its
497506
first event, so probe and coordinator slots interleave by transcript
498507
arrival. The settle path must reproduce that exact ordering. This scans
499508
the transcript once, assigning each lane (probe ``agent_id`` or the
500-
sentinel coordinator, opened by its first root ``tool_result``) its
509+
sentinel coordinator, opened by its first ROOT ``tool_result`` β€” a
510+
probe's own tool_result, classified by *probe_ids*, never opens it) its
501511
first-seen ordinal, then writes the probe ordinals back onto *trace* in
502512
place. Returns the coordinator's ordinal, or ``None`` when the root
503513
issued no tool_result (no coordinator lane).
@@ -514,6 +524,8 @@ def _assign_lane_slots(
514524
elif kind == "tool_result":
515525
if not CoordinatorTrace.is_root_event(payload):
516526
continue
527+
if str(payload.get("agent_id") or "") in probe_ids:
528+
continue
517529
key = coordinator_key
518530
else:
519531
continue
@@ -525,38 +537,45 @@ def _assign_lane_slots(
525537
return slots.get(coordinator_key)
526538

527539
@staticmethod
528-
def _build_coordinator_lines(records: list[dict[str, Any]]) -> list[TraceLine]:
529-
"""Project the root's ``tool_result`` events into coordinator trace lines.
530-
531-
The probes run in their OWN sessions, so a ``tool_result`` in THIS
532-
session's transcript is always a ROOT tool call β€” no agent-id filter is
533-
needed. Each becomes one secret-free :class:`CoordinatorTrace` digest line
534-
(the SAME projection the live streamer uses β€” DRY), so a reconciled
535-
coordinator lane is byte-identical to one that streamed live. Empty when
536-
the root spawned probes for everything (only ``sub_agent`` events).
540+
def _build_coordinator_lines(
541+
records: list[dict[str, Any]], probe_ids: set[str]
542+
) -> list[TraceLine]:
543+
"""Project the ROOT's ``tool_result`` events into coordinator trace lines.
544+
545+
A ``tool_result`` in this transcript is NOT always the root's: probe
546+
tool calls ride the same transcript stamped with the probe's
547+
``agent_id`` (the child loop inherits the parent's event_logger β€” #102
548+
corrected the #95 own-sessions premise), so any payload whose
549+
``agent_id`` is a probe lane is excluded here exactly as the live
550+
streamer excludes it. Each remaining event becomes one secret-free
551+
:class:`CoordinatorTrace` digest line (the SAME projection the live
552+
streamer uses β€” DRY), so a reconciled coordinator lane is byte-identical
553+
to one that streamed live. Empty when the root issued no tool calls.
537554
"""
538555
lines: list[TraceLine] = []
539556
for rec in records:
540557
if rec.get("type") != "tool_result":
541558
continue
542559
payload = rec.get("payload") or {}
560+
if str(payload.get("agent_id") or "") in probe_ids:
561+
continue
543562
if CoordinatorTrace.is_root_event(payload):
544563
lines.append(CoordinatorTrace.line(payload))
545564
return lines
546565

547566
@staticmethod
548567
def _build_results(
549-
run_id: str, records: list[dict[str, Any]]
568+
run_id: str, records: list[dict[str, Any]], probe_ids: set[str]
550569
) -> list[SearchResult]:
551570
"""Collect the run's result cards from every ``scg_results`` emit.
552571
553-
Transcript-as-transport (#95): the root calls ``scg_results`` once (the
554-
playbook's terminal-discipline step) carrying the discrete cards behind
555-
the answer; the api projects each entry onto a stable-id
556-
:class:`SearchResult` via :class:`ResultsProjection` (the SAME read the
557-
live stream uses, so ids β€” and thus dedup β€” agree). Multiple emits are
558-
concatenated in order; the stable id (``r-<run_id8>-<n>`` per emit) keeps
559-
a re-emit from the same call idempotent across live + settle.
572+
Transcript-as-transport (#95/#102): the root emits once before the
573+
synthesis; each probe may emit its own grounded cards. The api projects
574+
every entry onto a stable-id :class:`SearchResult` via
575+
:class:`ResultsProjection` (the SAME read the live stream uses, salted
576+
by the emitting probe's ``agent_id``, so ids β€” and thus dedup β€” agree).
577+
Emits are concatenated in transcript order; the stable id keeps a
578+
re-emit of the same call idempotent across live + settle.
560579
"""
561580
out: list[SearchResult] = []
562581
seen: set[str] = set()
@@ -566,7 +585,9 @@ def _build_results(
566585
payload = rec.get("payload") or {}
567586
if not ResultsProjection.is_results_event(payload):
568587
continue
569-
for item in ResultsProjection.parse(run_id, payload):
588+
agent_id = str(payload.get("agent_id") or "")
589+
emitter = agent_id if agent_id in probe_ids else None
590+
for item in ResultsProjection.parse(run_id, payload, emitter=emitter):
570591
if item.id in seen:
571592
continue
572593
seen.add(item.id)

β€Žapps/mewbo_api/src/mewbo_api/agentic_search/scg/run_streamer.pyβ€Ž

Lines changed: 73 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -242,10 +242,12 @@ class ResultsProjection:
242242
243243
The ``scg_results`` SessionTool is transcript-as-transport (#95): it validates
244244
+ echoes, the api projects. This static maps each emitted entry onto a
245-
:class:`SearchResult` with a STABLE id (``r-<run_id8>-<n>``) so the live
246-
stream and settle reconciliation mint the same ids β€” the dedup key that keeps
247-
a result from being emitted twice. Both the live ``_project`` and settle read
248-
through here (the ProbeTrace stance), so a result is byte-identical either way.
245+
STABLE id β€” ``r-<run_id8>-<n>`` for the root's emit, ``r-<run_id8>-<agent8>-<n>``
246+
for a probe's (#102: probes emit their own cards; the agent suffix keeps
247+
concurrent emitters collision-free) β€” so the live stream and settle
248+
reconciliation mint the same ids β€” the dedup key that keeps a result from
249+
being emitted twice. Both the live ``_project`` and settle read through here
250+
(the ProbeTrace stance), so a result is byte-identical either way.
249251
"""
250252

251253
@staticmethod
@@ -254,22 +256,32 @@ def is_results_event(payload: dict[str, Any]) -> bool:
254256
return str(payload.get("tool_id") or "") == "scg_results"
255257

256258
@staticmethod
257-
def parse(run_id: str, payload: dict[str, Any]) -> list[SearchResult]:
259+
def parse(
260+
run_id: str, payload: dict[str, Any], *, emitter: str | None = None
261+
) -> list[SearchResult]:
258262
"""Map the emit's entries onto stable-id :class:`SearchResult`s.
259263
264+
``emitter`` is the probe ``agent_id`` when the emit came from a probe
265+
lane (``None`` for the root / a legacy payload): it salts the stable id
266+
so two agents' emits never collide β€” and because the same transcript
267+
event carries the same ``agent_id`` live and at settle, the ids agree
268+
across both reads (the dedup invariant).
269+
260270
Drops any entry that fails the wire model (an ungrounded/malformed card)
261271
rather than failing the whole projection β€” the tool already validated on
262272
the way in, this is the lenient read side. Fields map honestly onto
263-
``SearchResult``; the entry ``confidence`` has no wire home, so it is
264-
folded into ``relevance`` ONLY when ``relevance`` was not supplied (never
265-
overwriting an explicit rank) β€” keeping the strongest available signal on
266-
the projected card so the settle metrics can read it.
273+
``SearchResult``: ``confidence`` is surfaced verbatim on the wire (#102)
274+
AND folded into ``relevance`` ONLY when ``relevance`` was not supplied
275+
(never overwriting an explicit rank) β€” keeping the strongest available
276+
signal on the projected card so the settle metrics can read it.
267277
"""
268278
tool_input = payload.get("tool_input")
269279
entries = tool_input.get("results") if isinstance(tool_input, dict) else None
270280
if not isinstance(entries, list):
271281
return []
272282
prefix = run_id[:8]
283+
if emitter:
284+
prefix = f"{prefix}-{emitter[:8]}"
273285
out: list[SearchResult] = []
274286
for n, entry in enumerate(entries):
275287
if not isinstance(entry, dict):
@@ -286,22 +298,25 @@ def _to_result(result_id: str, entry: dict[str, Any]) -> SearchResult:
286298
287299
``relevance`` carries the entry's explicit rank; absent (or 0), the
288300
entry's ``confidence`` is folded in as the best available signal so the
289-
card and the settle metrics aren't blind. ``confidence`` itself has no
290-
wire field, so it is not surfaced separately.
301+
card and the settle metrics aren't blind. ``confidence`` ALSO rides its
302+
own wire field verbatim (#102) so the console can render the emitting
303+
agent's per-card certainty beside the relevance rank.
291304
"""
292305
raw_kind = str(entry.get("kind") or "docs")
293306
if raw_kind not in _RESULT_KINDS:
294307
raise ValueError(f"unknown result kind: {raw_kind}")
295308
kind = cast(ResultKindLiteral, raw_kind)
296309
url = entry.get("url")
310+
raw_confidence = entry.get("confidence")
297311
relevance = float(entry.get("relevance") or 0.0)
298-
if relevance <= 0.0 and entry.get("confidence") is not None:
299-
relevance = float(entry.get("confidence") or 0.0)
312+
if relevance <= 0.0 and raw_confidence is not None:
313+
relevance = float(raw_confidence or 0.0)
300314
return SearchResult(
301315
id=result_id,
302316
source=str(entry.get("source") or ""),
303317
kind=kind,
304318
relevance=relevance,
319+
confidence=float(raw_confidence) if raw_confidence is not None else None,
305320
title=str(entry.get("title") or ""),
306321
url=str(url) if url else "",
307322
snippet=str(entry.get("snippet") or ""),
@@ -409,20 +424,58 @@ def _consume(self) -> None:
409424
def _project(self, record: EventRecord) -> None:
410425
"""Project one transcript event onto the run event log (live).
411426
412-
Two lanes share this seam: a probe's ``sub_agent`` lifecycle (the probes
413-
run in their OWN sessions, but the run streamer subscribes to THIS
414-
session's bus β€” so what arrives here as a ``sub_agent`` is the parent's
415-
view of its children) and the ROOT coordinator's ``tool_result`` events
416-
(#95 β€” the root agent's own tool activity on this session's bus). The
417-
probe tool calls never appear here; they live in the probes' sessions.
427+
Three event classes share this seam: a probe's ``sub_agent`` lifecycle
428+
(the parent's view of its children), the ROOT coordinator's
429+
``tool_result`` events (#95), and a PROBE's own ``tool_result`` events.
430+
The last class exists because a child loop INHERITS the parent's
431+
``event_logger`` (core ``AgentContext.child``) β€” probe tool calls land
432+
on THIS session's transcript/bus stamped with the probe's ``agent_id``
433+
(#102; the original #95 premise that they "live in the probes' own
434+
sessions" was wrong, verified live). A probe ``tool_result`` must
435+
therefore be classified by ``payload.agent_id`` against the known probe
436+
lanes (the spawn's ``sub_agent`` ``start`` always precedes the child's
437+
first tool call, so the lane is known by the time its tools fire) β€”
438+
only its ``scg_results`` emit projects (as the probe's result cards);
439+
everything else stays off the coordinator lane.
418440
"""
419441
kind = record.get("type")
420442
raw = record.get("payload")
421443
payload: dict[str, Any] = raw if isinstance(raw, dict) else {}
422444
if kind == "sub_agent":
423445
self._project_sub_agent(payload)
424446
elif kind == "tool_result":
425-
self._project_coordinator(payload)
447+
emitter = self._probe_emitter(payload)
448+
if emitter is not None:
449+
self._project_probe_tool(emitter, payload)
450+
else:
451+
self._project_coordinator(payload)
452+
453+
def _probe_emitter(self, payload: dict[str, Any]) -> str | None:
454+
"""The probe ``agent_id`` when this tool_result came from a probe lane.
455+
456+
``None`` for the root's own tool calls (its ``agent_id`` never opens a
457+
probe lane) and for legacy payloads with no ``agent_id`` β€” both keep
458+
the historical coordinator-lane path.
459+
"""
460+
agent_id = str(payload.get("agent_id") or "")
461+
if not agent_id:
462+
return None
463+
with self._lock:
464+
return agent_id if agent_id in self._agents else None
465+
466+
def _project_probe_tool(self, emitter: str, payload: dict[str, Any]) -> None:
467+
"""Project a probe's own ``tool_result`` (#102).
468+
469+
Only the probe's ``scg_results`` emit reaches the run log β€” its entries
470+
become ``result`` events with probe-salted stable ids. Every other probe
471+
tool call is deliberately dropped here: the probe lane stays
472+
lifecycle-only (#86 β€” its evidence rides the ``stop`` summary), and the
473+
coordinator lane must not absorb child activity (the #95 mislabel).
474+
"""
475+
if ResultsProjection.is_results_event(payload):
476+
self._emit_results(
477+
ResultsProjection.parse(self._run_id, payload, emitter=emitter)
478+
)
426479

427480
def _project_sub_agent(self, payload: dict[str, Any]) -> None:
428481
"""Project one probe ``sub_agent`` lifecycle event onto a probe lane."""

β€Žapps/mewbo_api/src/mewbo_api/agentic_search/schemas.pyβ€Ž

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,10 @@ class SearchResult(_Wire):
284284
source: str
285285
kind: ResultKindLiteral
286286
relevance: float = 0.0
287+
# How sure the emitting agent is this hit answers the query (0..1) β€”
288+
# carried verbatim from an ``scg_results`` entry (agent-emitted cards
289+
# only). ``None`` when the emitter offered no defensible confidence.
290+
confidence: float | None = None
287291
title: str
288292
url: str = ""
289293
snippet: str = ""

0 commit comments

Comments
Β (0)