Commit 18efd9b
feat(mcp-web-fetch): add @frontagent/mcp-web-fetch adapter package (unwired) (#358)
* feat(mcp-web-fetch): add @frontagent/mcp-web-fetch adapter package (unwired)
Adds a new workspace package providing an agent web_fetch MCP tool: fetch a
URL and return cleaned, readable text (HTML→text) for looking up library/
framework docs, API references, and error resolutions during planning and
execution.
Mirrors the @frontagent/mcp-filesense adapter shape (zero runtime deps,
ported engine, *Schema + handle*Tool dispatch, tools.security.test.ts).
Ports the SSRF-hardened fetch logic from the standalone ceilf6/websense
repo: url-safety guards (reject private/loopback/link-local/cloud-metadata
hosts, DNS-resolution check against rebinding, credential rejection),
per-redirect-hop re-validation, AbortController timeout, byte-cap, and a
pure HTML→text extractor.
This package is NOT yet wired into the agent's MCP client registries; that
is a follow-up PR. Part of #357 (incremental delivery, step 2 of 3).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(mcp-web-fetch): bind SSRF check to connection + enforce hard limits
Addresses repo-guard's two HIGH findings on the adapter:
1. DNS-rebinding (TOCTOU): the previous pre-resolve check did not bind to
the actual fetch connection, so a domain could resolve to a public IP
at check time and a private/metadata IP at connect time. Route fetch
through an undici Agent whose connect.lookup rejects private/loopback/
link-local resolved addresses at connection time, so the validated
address IS the connected address. Adds undici dependency (security-
justified). Synchronous parseAndValidateUrl pre-check is retained for
fast rejection of bad protocol/credentials/literal-private hosts.
2. Caller-amplifiable limits: maxBytes/timeoutMs/maxRedirects were soft
defaults a caller could overshoot. Add clampLimit() enforcing positive
integers capped at hard ceilings (bytes 5MB, timeout 60s, redirects 10)
in the engine, plus minimum/maximum constraints in the tool schema.
New tests for clampLimit and isConnectionAddressBlocked. Part of #357.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(mcp-web-fetch): block IPv4-mapped IPv6 SSRF bypass (hex & expanded)
repo-guard HIGH finding: isPrivateIpv6 only caught the dotted form
::ffff:127.0.0.1 and checked just the first hextet, so equivalents like
::ffff:7f00:1 (127.0.0.1), ::ffff:0a00:1 (10.0.0.1), and expanded
0:0:0:0:0:ffff:127.0.0.1 mapped to loopback/private IPv4 but slipped
through — an SSRF bypass shared by the pre-check and the connect-time
dispatcher.
Rewrite isPrivateIpv6 to fully normalize the address (expandIpv6 via
node:net isIP, embedded-IPv4-tail handling, :: expansion to 8 hextets),
then classify: ::/::1, IPv4-mapped and IPv4-compatible forms delegate to
isPrivateIpv4; fc00::/7 and fe80::/10 by mask. Adds regression tests for
hex/expanded mapped loopback/private forms and bracketed-host URLs.
Part of #357.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(mcp-web-fetch): add runtime SSRF/limit regression tests + dns all:false
Addresses repo-guard's remaining non-blocking notes:
- Force all:false in the safeDispatcher dns.lookup so the resolved address
is always a single string (not an array when undici passes all:true).
- Add an internal test-only fetchImpl injection seam to fetchUrl so the
runtime security paths can be verified deterministically without network.
- New engine.test.ts: redirect hop to 127.0.0.1 is rejected before the
second fetch; oversized body is truncated to maxBytes; maxRedirects cap
throws; happy-path HTML→text/title/status.
40/40 tests pass. Part of #357.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(mcp-web-fetch): export and directly test the dispatcher safeLookup
repo-guard asked to verify the ACTUAL connection-time DNS guard (the
function undici's dispatcher calls), not just the pure helper + mocked
fetch. Extract the dispatcher's inline lookup into an exported safeLookup
(behavior unchanged; matches Node's net.LookupFunction signature) and add
engine.lookup.test.ts asserting it rejects localhost/127.0.0.1/::1 (private,
resolved locally — no network) and allows 8.8.8.8. This proves the exact
lookup wired into the safe undici Agent blocks private resolved addresses.
44/44 tests. Part of #357.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>1 parent 6bda08e commit 18efd9b
17 files changed
Lines changed: 1150 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
0 commit comments