Skip to content

[gateway] DIFC-filtered tool results should include filtering metadata to prevent agent misinterpretation #4420

@lpcox

Description

@lpcox

Problem

When list_issues, search_issues, or issue_read results are entirely removed by DIFC integrity filtering, the tool response returned to the agent is semantically identical to a genuinely empty result — the agent has no way to tell "no items exist" from "items exist but were filtered." This causes agents to take incorrect follow-up actions, such as creating duplicate issues when their own prior work was filtered, or drifting into "scheduled mode" when a targeted-dispatch bound issue was filtered.

Two distinct user-reported bugs share this root cause:

  1. Targeted dispatch drift — A run bound to a specific issue via workflow_dispatch issue_number=N receives an empty list_issues response because #N is DIFC-filtered. The agent interprets [] as "no open issues," reclassifies itself as a scheduled run, triages unrelated items, and emits a false-success noop. (gh-aw#21784)

  2. Duplicate issue creation — Repo Assist searches for an existing monthly-activity issue it previously created; the result is DIFC-filtered to []; the agent concludes no issue exists and creates a duplicate. (Related to gh-aw#22533)

Source: github/gh-aw#21784

Analysis

The filtering is applied inside the DIFC evaluator and the response-labeling path in the server:

  • internal/difc/evaluator.goFilterCollection(): Returns a FilteredCollectionLabeledData struct that correctly tracks Accessible, Filtered, and TotalCount. The distinction already exists internally but is not surfaced to the agent.

  • internal/server/unified.go / internal/server/routed.go — The path that converts FilteredCollectionLabeledData back into a JSON-RPC tool response to forward to the LLM. The current implementation discards the Filtered slice and only returns Accessible items; no annotation is added.

  • guards/github-guard/rust-guard/src/lib.rsLabelResponse marks individual items as filtered with per-item reasons, but that information is not propagated back to the MCP caller as part of the tool result content.

The FilteredCollectionLabeledData struct already has Filtered []FilteredItemDetail and TotalCount int. The server just needs to inject a summary into the tool result when len(Filtered) > 0.

Proposed Solution

1. Inject a filtering notice into the tool result content when items are suppressed

In internal/server/unified.go (and routed.go) at the point where the DIFC-filtered response is serialized and forwarded to the agent, check whether filtered.FilteredCount > 0 and, if so, append a structured annotation to the tool result:

// After applying LabelResponse + FilterCollection:
if filtered.FilteredCount > 0 {
    // Append a "_difc_filter_notice" field to the JSON tool result content
    // OR prepend a system-level text notice to the result array
    notice := fmt.Sprintf(
        "[%d result(s) hidden by integrity policy — min-integrity=%s]",
        filtered.FilteredCount,
        policyMinIntegrity,
    )
    // Inject notice into the MCP ToolResult content
}

This gives the agent a reliable signal: list_issues returning [] with a notice is fundamentally different from list_issues returning [] with no notice.

2. For single-item reads (issue_read), return a structured error instead of empty

When issue_read is called and the single result is filtered, return an MCP error with a clear message:

Issue #N exists but is not accessible — filtered by integrity policy
(author_association: NONE, min-integrity: unapproved).

This prevents the agent from interpreting "filtered single-item read" as "issue does not exist."

3. Tests to add

In internal/server/unified_test.go (or integration tests in test/integration/):

  • list_issues with all items filtered → verify tool result contains _difc_filter_notice
  • issue_read with a filtered single item → verify MCP error is returned with integrity reason
  • list_issues with genuinely empty repo → verify no filter notice is present

Testing

  1. Configure gateway with min-integrity: approved on a public repo.
  2. Call list_issues where all open issues are authored by users with author_association: NONE.
  3. Verify the tool response contains a filtering notice alongside the empty array.
  4. Confirm an LLM agent receiving this response can distinguish it from a truly empty repo.
  5. Regression test: verify the licensee/licensee scenario from gh-aw#22533 no longer creates duplicates when Repo Assist's own bot-created issues are filtered.

Generated by Gateway Issue Dispatcher · ● 1.3M ·

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions