feat(web-search): add WebSearchTask with pluggable search providers

[sroussey]

Adds @workglow/web-search — one WebSearchTask serving both plain search APIs and model-grounded search behind a single normalized output shape, routed by provider capability.

Design spec and implementation plan live in the prd repo on the same branch name (docs/superpowers/specs/2026-08-22-web-search-task-design.md).

Why one task for both families

The obvious split — "search APIs" vs "grounded search" — isn't real. Tavily returns a synthesized answer when asked; Exa returns full page content; Anthropic's web_search returns an answer plus citations that are structurally {title, url, snippet}. So this is one shape whose fields different providers can or cannot populate — a capability matrix, not a discriminated union.

results is always present (for a grounded provider, those are its citations). answer is populated only when the caller asks for one, uniformly across providers.

Server-side only

Every commercial search API authenticates with a request header, which forces a CORS preflight none of them answer. Google's Custom Search JSON API is the one exception that genuinely permits cross-origin XHR — and it is closed to new customers and sunsets 2027-01-01. Independently, a browser-executed search would put the API key where any visitor can read it.

The browser entry therefore registers the task (so a builder UI can render the node and validate a graph) and no providers.

Capability routing

provider is required with no default, mirroring response_type on FetchUrlTask — which provider serves a request decides its cost, rate limit and quality. "auto" opts into routing over the requested options; the provider that ran is always reported on the provider output port. A pinned provider that cannot honor an option throws rather than silently rerouting.

domainFilter is three-valued: "native", "query-operator" (the task rewrites the query with site:), or false. Date filtering is never emulated — post-filtering by publishedDate breaks maxResults and drops every result whose date the provider omitted, so dateFilter: false means such a request is refused.

Providers

provider	auth	answer	content	domain filter	date filter
brave	X-Subscription-Token	no	no	via site:	yes
tavily	bearer	yes	yes	native	yes
searxng	none (self-hosted)	no	no	via site:	no
anthropic	vendor SDK	yes	no	native	no

HTTP adapters execute by owning a FetchUrlTask, inheriting credential resolution via credential_key, SafeFetch's redirect/SSRF checks, retry/backoff, per-attempt timeouts and the response cache.

They do not inherit the queue's rate limiter, and cannot: FetchUrlTask refuses credential_key on the queued path because a queued payload is persisted to durable storage, so every keyed provider runs inline. Search APIs are metered against hard monthly quotas, so bounding a MapTask fan-out remains the caller's responsibility. (This corrects an earlier overclaim in this PR, caught in review.)

The grounded adapter ships as a new ./web-search subpath on @workglow/anthropic, so @workglow/web-search has no dependency on @workglow/ai.

Two Anthropic traps handled and tested: a web_search_tool_result block carries a list on success and an error object on failure — at HTTP 200, raising nothing — so reading it unbranched records a quota failure as a search that found nothing; and a server-tool turn can stop with stop_reason: "pause_turn", which must be resumed or the answer truncates silently.

Tests

77 tests. SearXNG needs no key and has no quota, so it is the only provider whose integration test can run unmocked (.integration.test.ts, excluded from the default tier, skipped unless WEB_SEARCH_SEARXNG_URL is set). The rest are fixture-driven.

Known unrelated CI failure

test-vitest-ai-provider-api fails on PermanentJobError: Provider DEEPSEEK failed for StructuredGenerationTask: 402 Insufficient Balance. That is the DeepSeek account balance, not this diff — nothing here touches DeepSeek (the 31 changed files are the new package, the Anthropic subpath, and docs). It needs a top-up or a key rotation, not a code change.

Not in scope

No streaming (a grounded answer could stream, but results cannot arrive incrementally usefully). No re-ranking, deduplication, or multi-provider fan-out — one query, one provider, one response.

[Copilot]

Pull request overview

Introduces a new @workglow/web-search package that standardizes web search behind a provider interface with capability-based routing, and adds an Anthropic grounded web-search provider entrypoint in @workglow/anthropic.

Changes:

• Adds @workglow/web-search with WebSearchTask, provider registry, capability checks, and built-in HTTP providers (Brave, Tavily, SearXNG).
• Adds @workglow/anthropic/web-search entrypoint implementing the IWebSearchProvider contract.
• Adds tests and wiring (exports/build scripts/tsconfig references) to support the new package and provider.

Reviewed changes

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
providers/anthropic/tsconfig.json	Adds TS project reference to the new packages/web-search.
providers/anthropic/src/web-search/AnthropicWebSearchProvider.ts	Implements Anthropic grounded web-search provider via vendor SDK.
providers/anthropic/src/web-search.ts	Adds ./web-search entrypoint and registration helper for the Anthropic provider.
providers/anthropic/src/tests/AnthropicWebSearchProvider.test.ts	Unit tests for Anthropic web-search provider behavior (tool type, mapping, pause_turn, domain lists).
providers/anthropic/package.json	Exports ./web-search, builds it, and adds dependency on @workglow/web-search.
packages/web-search/tsconfig.json	Adds composite TS config for the new @workglow/web-search package.
packages/web-search/src/WebSearchTask.ts	Implements the WebSearchTask with provider routing, capability enforcement, and request adaptation.
packages/web-search/src/WebSearchProviderRegistry.ts	Adds a process-wide provider registry with routing and helpful error messages.
packages/web-search/src/queryOperators.ts	Implements site: query rewriting for providers that only support domain filtering via query operators.
packages/web-search/src/providers/TavilyWebSearchProvider.ts	Adds Tavily HTTP adapter with capability declaration and request/response mapping.
packages/web-search/src/providers/SearxngWebSearchProvider.ts	Adds SearXNG adapter with base URL validation and result normalization.
packages/web-search/src/providers/httpSearch.ts	Shared helper to execute provider HTTP calls via an owned FetchUrlTask.
packages/web-search/src/providers/BraveWebSearchProvider.ts	Adds Brave HTTP adapter including date-range mapping to freshness.
packages/web-search/src/node.ts	Node entrypoint that registers the task and built-in providers on import.
packages/web-search/src/IWebSearchProvider.ts	Defines provider interface, request/response types, and capability model.
packages/web-search/src/common.ts	Common exports plus registration helpers for task and built-in providers.
packages/web-search/src/capabilityCheck.ts	Implements capability/request gap detection used for routing and pinned-provider validation.
packages/web-search/src/browser.ts	Browser entrypoint that registers only the task (no providers).
packages/web-search/src/tests/WebSearchTask.test.ts	Tests routing behavior, pinned-provider validation, query rewriting, and maxResults clamping.
packages/web-search/src/tests/WebSearchProviderRegistry.test.ts	Tests registry semantics (register/get/route/require/clear) and error messages.
packages/web-search/src/tests/TavilyWebSearchProvider.test.ts	Tests Tavily adapter mapping, auth scheme, and option behavior.
packages/web-search/src/tests/SearxngWebSearchProvider.test.ts	Tests SearXNG adapter URL construction, credential suppression, normalization, and truncation.
packages/web-search/src/tests/SearxngWebSearchProvider.integration.test.ts	Optional live integration test for SearXNG behind env var gating.
packages/web-search/src/tests/queryOperators.test.ts	Tests site: query operator rewriting and normalization.
packages/web-search/src/tests/entries.test.ts	Tests that entrypoints register tasks/providers as intended.
packages/web-search/src/tests/capabilityCheck.test.ts	Tests capability gap detection across request options.
packages/web-search/src/tests/BraveWebSearchProvider.test.ts	Tests Brave adapter mapping, auth header scheme, and freshness date mapping.
packages/web-search/README.md	Documents server-only behavior, provider matrix, and usage patterns.
packages/web-search/package.json	Defines new package exports/build/test scripts and peer deps.
bun.lock	Adds workspace entry and dependency wiring for @workglow/web-search.
.claude/CLAUDE.md	Updates repo architecture and adds @workglow/web-search documentation section.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.