feat(research): page reads go through codeNERD's browser, not net/http

Steve 2026-09-29: use the browser capabilities native to codeNERD to scrape and research.
web_fetch, web_search and context7_fetch now read through one browser page read (isolated tab,
load + DOM stability, rendered HTML/text, tab always closed); research.browser_headless (default
true) keeps reads in a separate headless Chrome. A missing Chrome is an error naming
browser.launch. The kernel web_search action's Target becomes the query. The Context7 key is no
longer sent to raw GitHub.

getBrowserManager no longer builds a hidden nil-kernel manager. Root review fix: the nine
browser tools that dereferenced it would now have panicked unbound; each refuses at entry with
requireBoundBrowser (TestBrowserToolsRefuseWhenUnbound) and evidence recording is nil-safe.

Implemented by a Grok lane (R2); verified in an isolated tree (HEAD b069d3e + this change):
build ok; research (real Chrome), config, core, core/defaults(+policy), browser ok. Live
DuckDuckGo not exercised. Spec: Docs/architecture/tools/13-RESEARCH-READS-THROUGH-BROWSER.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kLAKtMKJieinizH6Zjx2b

Author: theRebelliousNerd

Docs/architecture/tools/13-RESEARCH-READS-THROUGH-BROWSER.md

@@ -0,0 +1,60 @@
+---
+doc-class: shipped
+subsystem: tools
+implementation-status: shipped
+last-verified: 2026-09-29
+verified-against: b069d3e4
+supersedes: []
+---
+
+# 13 — Research reads pages through codeNERD's browser
+
+> Written to `Docs/journeys/09-architecture-doc-standard.md` beside this package's July-2026
+> files, which are not rewritten yet. Question 2 (what runs) for one capability, written after
+> the code (work in flight on 2026-09-29).
+
+## What runs
+
+Steve, 2026-09-29: "instead of fetch for web page pulling or whatever we are using, use the
+browser capabilities native to codenerd to scrape data and do research." No research tool reads
+a web page with `net/http` any more.
+
+- **One page read.** `readBrowserPage` (`internal/tools/research/browser_read.go:205`) opens an
+  isolated tab, navigates, waits for load and then for DOM stability (the same stable-wait the
+  `browser_wait` tool uses), reads the rendered HTML and text, and always closes the tab. A page
+  that never goes quiet is an error with no body, never a blank-frame extract. A non-2xx document
+  response is reported as `HTTP <code>`.
+- **Headless by default.** `research.browser_headless` (`internal/config/research.go:52`, default
+  true at `internal/config/research.go:69`) runs research reads in a separate headless Chrome
+  (`researchHeadlessManager`, `internal/tools/research/browser_read.go:139`), so a read never pops
+  a window even when the interactive browser (`browser.headless`) is visible. A missing Chrome is
+  an error naming `browser.launch`.
+- **The three tools.** `web_fetch` (`internal/tools/research/web_fetch.go:56`) reads through the
+  browser, then keeps its HTML-to-markdown conversion and paging. `web_search`
+  (`internal/tools/research/web_search.go:47`) drives the DuckDuckGo HTML results page in the
+  browser and parses the live DOM. `context7_fetch` (`internal/tools/research/context7.go:43`)
+  reads each documentation URL through the browser; the Context7 key is no longer sent to raw
+  GitHub, where it was never a credential.
+- **No hidden second browser.** `getBrowserManager` (`internal/tools/research/browser.go`) returns
+  the manager boot bound with `SetBrowserRuntime`, or nil. Every browser tool refuses at its entry
+  with `requireBoundBrowser` when nothing is bound (`TestBrowserToolsRefuseWhenUnbound`).
+- **Kernel action.** A kernel `web_search` action's `Target` becomes the query
+  (`internal/core/virtual_store_actions.go`; `TestHandleModularTool_WebSearchTargetIsQuery`).
+
+Search discovery on Meta is `grounded_web_search`
+(`Docs/architecture/perception/13-META-SEARCH-GROUNDING.md`); the browser then reads the pages
+it returns.
+
+## Evidence
+
+`internal/tools/research/browser_read_test.go` drives real Chrome against local servers: static
+HTML, a page that inserts its content by script after load, `text/plain` markdown, tab count back
+to baseline after each read, and no `net/http` use in the research tools. It ran (Chrome found at
+the standard Windows path) on 2026-09-29.
+
+## Not verified / open
+
+- The live DuckDuckGo site was not opened; headless Chrome may receive a challenge page there.
+  The parser is proven on a local document with the same markup.
+- A custom `browser.launch` does not reach the headless research browser until the factory
+  passes it (`internal/system/factory.go`); auto-detected Chrome is used.

internal/config/research.go

@@ -13,16 +13,16 @@ import (
 //
 // Why its own section instead of llm_timeouts: those bound LLM calls (one
 // HTTP call to a model vendor, a slot wait, an articulation call). These
-// bound researcher HTTP fetches and browser evidence paging — different
+// bound research browser page reads and browser evidence paging — different
 // vendors, different latency profiles, different owners. Every value here
 // bounds one request or one paging window, never a run: a windowed result
 // names the remainder and the offset that reaches it.
 type ResearchConfig struct {
-	// WebFetchTimeout bounds one web_fetch HTTP round trip.
+	// WebFetchTimeout bounds one web_fetch browser page read.
 	WebFetchTimeout string `json:"web_fetch_timeout,omitempty"`
-	// Context7Timeout bounds one context7_fetch HTTP round trip.
+	// Context7Timeout bounds one context7_fetch browser page read.
 	Context7Timeout string `json:"context7_timeout,omitempty"`
-	// WebSearchTimeout bounds one web_search HTTP round trip.
+	// WebSearchTimeout bounds one web_search browser page read.
 	WebSearchTimeout string `json:"web_search_timeout,omitempty"`
 	// BrowserExtractTimeout bounds one browser_extract DOM read.
 	BrowserExtractTimeout string `json:"browser_extract_timeout,omitempty"`
@@ -43,6 +43,13 @@ type ResearchConfig struct {
 	// view when max_items is absent: compact stays the cheap rung of the
 	// summary/compact/full ladder.
 	BrowserReasonCompactItems int `json:"browser_reason_compact_items,omitempty"`
+
+	// BrowserHeadless is whether research page reads (web_fetch, web_search,
+	// context7_fetch) launch their own headless Chrome. Default true: a read
+	// does not pop a window when browser.headless is false. False uses the
+	// boot-bound browser manager, which may be headed. A pointer so an absent
+	// key stays the default rather than false.
+	BrowserHeadless *bool `json:"browser_headless,omitempty"`
 }
 
 // DefaultResearchConfig is the research section with every field written
@@ -59,6 +66,7 @@ func DefaultResearchConfig() ResearchConfig {
 		BrowserExtractMaxCharsCap: 32000,
 		BrowserReasonItems:        20,
 		BrowserReasonCompactItems: 10,
+		BrowserHeadless:           boolPtr(true),
 	}
 }
 
@@ -98,6 +106,11 @@ func (c ResearchConfig) WithDefaults() ResearchConfig {
 	if c.BrowserReasonCompactItems == 0 {
 		c.BrowserReasonCompactItems = d.BrowserReasonCompactItems
 	}
+	if c.BrowserHeadless == nil {
+		// Copy the bool. Aliasing d.BrowserHeadless would let a caller mutate
+		// the process default through the returned struct.
+		c.BrowserHeadless = boolPtr(*d.BrowserHeadless)
+	}
 	return c
 }
 
@@ -114,6 +127,7 @@ type ResearchPolicy struct {
 	BrowserExtractMaxCharsCap int
 	BrowserReasonItems        int
 	BrowserReasonCompactItems int
+	BrowserHeadless           bool
 }
 
 // Resolve defaults, checks and parses the section. Every problem Check
@@ -131,6 +145,10 @@ func (c ResearchConfig) Resolve() (ResearchPolicy, error) {
 		v, _ := time.ParseDuration(s) // Check parsed every one of these
 		return v
 	}
+	headless := true
+	if c.BrowserHeadless != nil {
+		headless = *c.BrowserHeadless
+	}
 	return ResearchPolicy{
 		WebFetchTimeout:           d(c.WebFetchTimeout),
 		Context7Timeout:           d(c.Context7Timeout),
@@ -140,6 +158,7 @@ func (c ResearchConfig) Resolve() (ResearchPolicy, error) {
 		BrowserExtractMaxCharsCap: c.BrowserExtractMaxCharsCap,
 		BrowserReasonItems:        c.BrowserReasonItems,
 		BrowserReasonCompactItems: c.BrowserReasonCompactItems,
+		BrowserHeadless:           headless,
 	}, nil
 }
 

internal/config/research_test.go

@@ -21,14 +21,28 @@ func TestDefaultResearchConfig(t *testing.T) {
 		got.BrowserReasonItems != 20 || got.BrowserReasonCompactItems != 10 {
 		t.Errorf("default windows = %+v", got)
 	}
+	if got.BrowserHeadless == nil || !*got.BrowserHeadless {
+		t.Errorf("default browser_headless = %v, want true", got.BrowserHeadless)
+	}
+}
+
+// ResearchConfig carries a *bool, so two default structs are not comparable
+// by address. The resolved policy is the value the tools actually read.
+func resolvedResearch(t *testing.T, c ResearchConfig) ResearchPolicy {
+	t.Helper()
+	p, err := c.Resolve()
+	if err != nil {
+		t.Fatalf("Resolve: %v", err)
+	}
+	return p
 }
 
 func TestGetResearchConfig_Overlay(t *testing.T) {
 	var nilCfg *UserConfig
-	if got := nilCfg.GetResearchConfig(); got != DefaultResearchConfig() {
+	if got := resolvedResearch(t, nilCfg.GetResearchConfig()); got != resolvedResearch(t, DefaultResearchConfig()) {
 		t.Errorf("nil config = %+v, want defaults", got)
 	}
-	if got := (&UserConfig{}).GetResearchConfig(); got != DefaultResearchConfig() {
+	if got := resolvedResearch(t, (&UserConfig{}).GetResearchConfig()); got != resolvedResearch(t, DefaultResearchConfig()) {
 		t.Errorf("absent section = %+v, want defaults", got)
 	}
 	cfg := &UserConfig{Research: &ResearchConfig{
@@ -48,6 +62,18 @@ func TestGetResearchConfig_Overlay(t *testing.T) {
 		got.BrowserReasonItems != 33 || got.BrowserReasonCompactItems != 44 {
 		t.Errorf("set section = %+v, want the set values", got)
 	}
+	if got.BrowserHeadless == nil || !*got.BrowserHeadless {
+		t.Errorf("absent browser_headless = %v, want true", got.BrowserHeadless)
+	}
+	off := false
+	explicit := (&UserConfig{Research: &ResearchConfig{BrowserHeadless: &off}}).GetResearchConfig()
+	if explicit.BrowserHeadless == nil || *explicit.BrowserHeadless {
+		t.Errorf("explicit false browser_headless = %v", explicit.BrowserHeadless)
+	}
+	resolved, err := explicit.Resolve()
+	if err != nil || resolved.BrowserHeadless {
+		t.Fatalf("explicit false resolved browser_headless = %v, %v", resolved.BrowserHeadless, err)
+	}
 }
 
 func TestResearchConfig_Resolve(t *testing.T) {
@@ -64,6 +90,9 @@ func TestResearchConfig_Resolve(t *testing.T) {
 	if p.BrowserExtractMaxChars != 8000 || p.BrowserReasonCompactItems != 10 {
 		t.Errorf("absent windows did not take defaults: %+v", p)
 	}
+	if !p.BrowserHeadless {
+		t.Errorf("absent browser_headless = false, want true")
+	}
 	if _, err := (ResearchConfig{WebFetchTimeout: "bogus"}).Resolve(); err == nil {
 		t.Error("Resolve accepted an unparseable duration")
 	}
@@ -122,8 +151,8 @@ func TestLoadUserConfig_InstallsResearchPolicy(t *testing.T) {
 	got := ResolvedResearchPolicy()
 	if got.WebFetchTimeout != 61*time.Second || got.BrowserExtractMaxChars != 111 ||
 		got.BrowserExtractMaxCharsCap != 222 || got.BrowserReasonItems != 33 ||
-		got.BrowserReasonCompactItems != 44 {
-		t.Errorf("installed policy = %+v, want the set values", got)
+		got.BrowserReasonCompactItems != 44 || !got.BrowserHeadless {
+		t.Errorf("installed policy = %+v, want the set values and default headless", got)
 	}
 	if _, err := LoadUserConfig(filepath.Join(dir, "missing.json")); err != nil {
 		t.Fatalf("LoadUserConfig (missing): %v", err)
@@ -144,3 +173,17 @@ func TestLoadUserConfig_RefusesBadResearch(t *testing.T) {
 		t.Errorf("LoadUserConfig error = %v, want research.web_fetch_timeout named", err)
 	}
 }
+
+func TestLoadUserConfig_ResearchBrowserHeadlessFalse(t *testing.T) {
+	t.Cleanup(func() { SetResearchPolicy(mustResolveResearchDefaults()) })
+	path := filepath.Join(t.TempDir(), "config.json")
+	if err := os.WriteFile(path, []byte(`{"research":{"browser_headless":false}}`), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := LoadUserConfig(path); err != nil {
+		t.Fatalf("LoadUserConfig: %v", err)
+	}
+	if got := ResolvedResearchPolicy(); got.BrowserHeadless {
+		t.Fatalf("installed browser_headless = true, want false")
+	}
+}

internal/core/virtual_store_actions.go

@@ -822,6 +822,12 @@ func (v *VirtualStore) handleModularTool(ctx context.Context, req ActionRequest)
 		args["topic"] = req.Target
 	case ActionWebFetch:
 		args["url"] = req.Target
+	case ActionWebSearch:
+		// Kernel actions carry the search string in Target. An explicit
+		// target wins; an empty target still lets a payload query through.
+		if req.Target != "" {
+			args["query"] = req.Target
+		}
 	case ActionBrowserNavigate:
 		args["url"] = req.Target
 		if sid, ok := req.Payload["session_id"].(string); ok {

internal/core/virtual_store_web_search_target_test.go

@@ -0,0 +1,55 @@
+package core
+
+import (
+	"context"
+	"testing"
+
+	"codenerd/internal/tools"
+)
+
+func TestHandleModularTool_WebSearchTargetIsQuery(t *testing.T) {
+	vs, _ := createActionsTestVS(t)
+	var got map[string]any
+	err := vs.modularTools.Register(&tools.Tool{
+		Effect:   tools.EffectRead,
+		Name:     "web_search",
+		Category: tools.CategoryResearch,
+		Execute: func(ctx context.Context, args map[string]any) (string, error) {
+			got = args
+			return "ok", nil
+		},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	res, err := vs.handleModularTool(context.Background(), ActionRequest{
+		Type:    ActionWebSearch,
+		Target:  "rod browser",
+		Payload: map[string]any{"max_results": 3, "query": "ignored"},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !res.Success {
+		t.Fatalf("handleModularTool: %+v", res)
+	}
+	if got["query"] != "rod browser" {
+		t.Fatalf("query = %#v, want the action target", got["query"])
+	}
+	if got["max_results"] != 3 {
+		t.Fatalf("max_results = %#v, want the payload value", got["max_results"])
+	}
+
+	got = nil
+	res, err = vs.handleModularTool(context.Background(), ActionRequest{
+		Type:    ActionWebSearch,
+		Payload: map[string]any{"query": "from-payload"},
+	})
+	if err != nil || !res.Success {
+		t.Fatalf("empty target: %v %+v", err, res)
+	}
+	if got["query"] != "from-payload" {
+		t.Fatalf("empty target query = %#v, want the payload query", got["query"])
+	}
+}

internal/tools/research/agents.md

@@ -27,3 +27,10 @@ Do not change browser authority, manager lifetime, or other sessions from
 this tool. Keep each result bounded (one window); never return unbounded DOM
 or HTML — but never silently drop content either: anything past the window
 must be named with the offset that reaches it.
+
+Page reads go through the browser (2026-09-29): `web_fetch`, `web_search` and
+`context7_fetch` call `readBrowserPage` (isolated tab, load + DOM stability,
+always closed), headless unless `research.browser_headless` is false. Do not
+add a `net/http` page read. Browser tools refuse with `requireBoundBrowser`
+when boot bound no manager; there is no fallback manager. Spec:
+`Docs/architecture/tools/13-RESEARCH-READS-THROUGH-BROWSER.md`.