Improve watch dependency coverage and simplify settings rebuilds

.gitignore

@@ -23,3 +23,4 @@ test-results
 *.d.ts.map
 !types/**/*.d.ts
 !types/**/*.d.ts.map
+test-cases/generated-pages/.streaming

docs/data/README.md

@@ -433,11 +433,12 @@ Helpers that process `changes` can use the exported `GlobalDataChanges` type.
 
 - State lasts for one watch session, not across process restarts.
   Always handle `changes.kind === 'reset'`, including after failed builds.
-- Markdown and HTML source edits can update individual entries.
-  Changes to watched modules, including JavaScript/TypeScript pages, helpers, browser entries, and settings, reset the saved index and rebuild the site.
+- Source-page edits and tracked helper changes can update individual entries.
+  Changes to `global.data.*` or settings inputs reset the saved index.
 - Store structured-cloneable values such as plain records, arrays, or maps—not `PageData` instances, functions, or native objects with shared mutable storage.
 - A page can appear in `changes.upserted` even when its content is unchanged.
   Replace its cached entry, or delete it if the page no longer belongs in your index.
-- Restart watch mode after changing imported JSON, files read with `fs.readFile()`, environment variables, or other inputs outside page dependency tracking.
+- Relative static imports within the watched source tree are tracked, including imported JSON.
+  Restart watch mode after changing files read with `fs.readFile()`, environment variables, or other inputs outside page dependency tracking.
 - Static re-exports (`export … from`) are not followed by dependency tracking ([#328](https://github.com/bcomnes/domstack/issues/328)).
   Use an explicit import followed by a local export, or restart watch mode after those dependencies change.

docs/implementation/README.md

@@ -229,7 +229,7 @@ Require a full page retry`"]
 
 Bundle replanning returns only a page plan or a skip; it does not restart esbuild again.
 After a page-build failure, the next page-producing plan retries the complete page phase rather than trusting incremental filters.
-Manifest-settings changes and service-worker entry additions or removals skip page rendering unless an incremental global-data index needs to be reset.
+Manifest-settings changes and service-worker entry additions or removals skip page rendering unless the module also has server-side consumers.
 The one-shot manifest pipeline is not part of watch execution.
 
 > [!NOTE]
@@ -245,14 +245,11 @@ DOMStack uses these rebuild scopes:
 - **Full page/template rebuild**: DOMStack renders every source-backed and generated page and every template without restarting esbuild.
 - **Full rebuild**: DOMStack rediscovers the source tree, restarts esbuild, renders all pages and templates, and refreshes its dependency maps.
 
-Like templates, generated-pages modules rebuild when their own source or watched imported dependencies change.
+Like templates, generated-pages modules rebuild when their own source or imported dependencies change.
 When a targeted build recomputes global data, DOMStack compares top-level values with the previous successful build and adds only subscribers of changed keys to the rebuild set.
 
 ### What triggers what
 
-The tables below apply when `global.data.ts` does not save state with `setState`.
-With an [incremental index](../data/#incremental-global-data), Markdown and HTML edits can update individual entries, while changes to watched modules reset the index and trigger a **full rebuild**.
-
 | Change | Rebuild scope |
 |---|---|
 | Existing `page.ts`, `page.html`, `page.md`, or adjacent `page.vars.ts` | That page, plus subscribers of any changed global-data keys |
@@ -261,9 +258,8 @@ With an [incremental index](../data/#incremental-global-data), Markdown and HTML
 | Existing `*.template.ts` or a module it imports | Affected templates |
 | Existing `*.pages.ts` | Generated outputs owned by that file, then refresh dependency maps |
 | A module imported by `*.pages.ts` | Generated outputs owned by the importing files, then refresh dependency maps |
-| `markdown-it.settings.ts` | All source-backed Markdown pages, plus subscribers of any changed global-data keys |
 | `global.data.ts` | Consumers subscribed to top-level keys whose values changed |
-| `global.vars.ts` or `esbuild.settings.ts` | Full rebuild |
+| `global.vars.ts`, `markdown-it.settings.ts`, `esbuild.settings.ts`, or their tracked imports | Full rebuild and reset of retained global-data state |
 | `domstack-manifest.settings.ts` | No rebuild. The manifest pipeline is disabled in watch mode |
 | Existing client, style, Web Worker, or service-worker entry | esbuild only, unless the same module also has server-side consumers |
 | Static asset under `src` or a file under a `--copy` directory | cpx2 copies or removes the output directly |
@@ -283,7 +279,7 @@ When a full page/template rebuild or targeted generated-pages rebuild no longer
 ### Dependency tracking
 
 DOMStack uses [`@11ty/dependency-tree-typescript`](https://github.com/11ty/dependency-tree-typescript) to statically analyze ESM imports.
-Imported JSON changes do not trigger page rebuilds; restart watch mode after editing those files.
+The page watcher observes tracked imports within the source tree, including JSON files.
 DOMStack maintains dependency maps for:
 
 - Layout dependencies, source-backed pages using each layout, and generated-page owner layout membership
@@ -298,8 +294,8 @@ Dependency analysis is best-effort.
 When DOMStack cannot safely determine a targeted scope, it falls back to a broader rebuild or skips an unrelated changed module.
 
 esbuild tracks browser-entry dependencies independently.
-A module imported only by `client.ts` normally triggers rebundling without rendering page HTML.
-If `global.data.ts` has saved an incremental index, the module edit also resets that index and triggers a full rebuild.
+A module imported only by `client.ts` triggers rebundling without rendering page HTML.
+If the module also has server-side consumers, DOMStack rebuilds those consumers too.
 
 ### Stable entry filenames
 
@@ -326,7 +322,7 @@ Page HTML points to stable entry files during watch mode. esbuild can update an
 ### Manifest behavior
 
 Watch mode builds and rebundles the site service worker, but it does not finalize, return, or write the [DOMStack manifest](../../docs/workers/#domstack-manifest).
-Editing `domstack-manifest.settings.ts` only triggers a rebuild when an incremental global-data index needs to be reset; it does not generate a manifest.
+Editing `domstack-manifest.settings.ts` does not trigger a watch rebuild unless it is also imported by server-side code.
 
 Use `domstack --serve` when testing manifest-driven cache behavior.
 It runs a one-shot build and serves the result without watch-mode filenames or live-reload HTML injection.

index.js

@@ -105,10 +105,13 @@ export class DomStack {
   #pagesFileDepMap = new Map()
   /** @type {Set<string>} Imported inputs of global.data, including its entry file. */
   #globalDataDepPaths = new Set()
-
+  /** @type {Set<string>} Settings roots and imports always require a full rebuild. */
+  #settingsDepPaths = new Set()
+  #dependencyAnalysisFailed = false
   /** @type {Set<string>} absolute filepaths of esbuild entry points */
   #esbuildEntryPoints = new Set()
-
+  /** @type {Set<string>} Known browser-only helpers can skip the page phase. */
+  #esbuildDepPaths = new Set()
   /** @type {Map<string, Set<string>>} source page or *.pages.* filepath → owned absolute output paths */
   #pageOutputMap = new Map()
   /** @type {PageOutputCache} Successful writes, including those before an iterator failure. */
@@ -335,7 +338,9 @@ export class DomStack {
     const anymatch = (/** @type {string} */name) => ig.ignores(relname(this.#src, name))
 
     const watcher = chokidar.watch(this.#src, {
-      ignored: (filePath, stats) => anymatch(filePath) || Boolean(stats?.isFile() && !isProcessedFile(filePath)),
+      // Observe non-page extensions too (for example statically imported JSON).
+      // Route only processed files and known dependencies after maps are ready.
+      ignored: filePath => anymatch(filePath),
       persistent: true,
       ignoreInitial: true,
       // Increase the atomic write window so editors that do slow atomic saves
@@ -453,21 +458,43 @@ export class DomStack {
       pagesFileDepMap: this.#pagesFileDepMap,
       pagesFileLayoutMap: this.#pagesFileLayoutMap,
       globalDataDepPaths: this.#globalDataDepPaths,
-      hasGlobalDataState: this.#watchSession?.globalDataBaseline?.state !== undefined,
+      settingsDepPaths: this.#settingsDepPaths,
+      dependencyAnalysisFailed: this.#dependencyAnalysisFailed,
       pageBuildFailed: this.#pageBuildFailed,
       esbuildEntryPoints: this.#esbuildEntryPoints,
+      esbuildDepPaths: this.#esbuildDepPaths,
     }
   }
 
+  /** @param {WatchEvent[]} events */
+  #filterWatchEvents (events) {
+    // Unknown inputs may be needed to recover after a failed build or analysis.
+    if (this.#pageBuildFailed || this.#dependencyAnalysisFailed) return events
+
+    const dependencies = [
+      this.#globalDataDepPaths,
+      this.#settingsDepPaths,
+      this.#layoutDepMap,
+      this.#pageDepMap,
+      this.#templateDepMap,
+      this.#pagesFileDepMap,
+      this.#esbuildDepPaths,
+    ]
+    return events.filter(({ filepath }) =>
+      isProcessedFile(filepath) || dependencies.some(paths => paths.has(filepath))
+    )
+  }
+
   /** @param {WatchEvent[]} events */
   async #handleWatchBatch (events) {
     const snapshot = this.#watchSnapshot()
     if (!snapshot) return
+    events = this.#filterWatchEvents(events)
     const event = events[0]
     if (!event) return
     const { plan, inputChanges } = planWatchBatch(snapshot, events)
-    // Preserve single-event bundle routing unless retained state needs a reset.
-    const singlePlan = !(snapshot.hasGlobalDataState && inputChanges.resetReason) &&
+    // Bundle replanning can skip page work, so it must not bypass a required reset.
+    const singlePlan = inputChanges.resetReason === undefined &&
       events.length === 1 && event.type !== 'change' && event.convention?.bundleScope
       ? planWatchEvent(snapshot, event)
       : null
@@ -703,15 +730,27 @@ export class DomStack {
     const pageDepMap = /** @type {Map<string, Set<PageInfo>>} */ (new Map())
     const templateDepMap = /** @type {Map<string, Set<TemplateInfo>>} */ (new Map())
     const pagesFileDepMap = /** @type {Map<string, Set<PagesFileInfo>>} */ (new Map())
-    const globalDataDepPaths = new Set()
-    if (siteData.globalData) {
-      globalDataDepPaths.add(siteData.globalData.filepath)
-      try {
-        for (const dep of await find(siteData.globalData.filepath)) globalDataDepPaths.add(resolve(dep))
-      } catch {
-        // Static import analysis is best-effort, as for page and layout helpers.
+    let dependencyAnalysisFailed = false
+    /** @param {...(string | undefined)} filepaths */
+    const rootDependencies = async (...filepaths) => {
+      const paths = new Set(/** @type {string[]} */ ([]))
+      for (const filepath of filepaths) {
+        if (!filepath) continue
+        paths.add(resolve(filepath))
+        try {
+          for (const dep of await find(filepath)) paths.add(resolve(dep))
+        } catch {
+          dependencyAnalysisFailed = true
+        }
       }
+      return paths
     }
+    const globalDataDepPaths = await rootDependencies(siteData.globalData?.filepath)
+    const settingsDepPaths = await rootDependencies(
+      siteData.globalVars?.filepath,
+      siteData.markdownItSettings?.filepath,
+      siteData.esbuildSettings?.filepath
+    )
 
     // layoutFileMap: layout filepath → layoutName
     for (const layout of Object.values(siteData.layouts)) {
@@ -728,7 +767,7 @@ export class DomStack {
           layoutDepMap.get(absPath)?.add(layout.layoutName)
         }
       } catch {
-        // dep analysis is best-effort
+        dependencyAnalysisFailed = true
       }
     }
 
@@ -748,7 +787,7 @@ export class DomStack {
 
     // pageDepMap: dep filepath → Set<PageInfo>
     for (const pageInfo of siteData.pages) {
-      const filesToTrack = [pageInfo.pageFile.filepath]
+      const filesToTrack = /\.[cm]?[jt]sx?$/.test(pageInfo.pageFile.filepath) ? [pageInfo.pageFile.filepath] : []
       if (pageInfo.pageVars) filesToTrack.push(pageInfo.pageVars.filepath)
       for (const file of filesToTrack) {
         try {
@@ -759,7 +798,7 @@ export class DomStack {
             pageDepMap.get(absPath)?.add(pageInfo)
           }
         } catch {
-          // best-effort
+          dependencyAnalysisFailed = true
         }
       }
     }
@@ -774,7 +813,7 @@ export class DomStack {
           templateDepMap.get(absPath)?.add(templateInfo)
         }
       } catch {
-        // best-effort
+        dependencyAnalysisFailed = true
       }
     }
 
@@ -788,6 +827,7 @@ export class DomStack {
           pagesFileDepMap.get(absPath)?.add(pagesFileInfo)
         }
       } catch (err) {
+        dependencyAnalysisFailed = true
         const message = err instanceof Error ? err.message : String(err)
         this.#logger.debug(`Could not analyze dependencies for pages file "${pagesFileInfo.pagesFile.relname}": ${message}`)
       }
@@ -804,6 +844,16 @@ export class DomStack {
       for (const asset of layoutBundleAssets(layout)) esbuildEntryPoints.add(resolve(asset.filepath))
     }
 
+    const esbuildDepPaths = new Set(/** @type {string[]} */ ([]))
+    for (const filepath of esbuildEntryPoints) {
+      if (!/\.[cm]?[jt]sx?$/.test(filepath)) continue
+      try {
+        for (const dep of await find(filepath)) esbuildDepPaths.add(resolve(dep))
+      } catch {
+        // Unknown browser helpers still take the conservative reset path.
+      }
+    }
+
     this.#layoutDepMap = layoutDepMap
     this.#layoutPageMap = layoutPageMap
     this.#pageFileMap = pageFileMap
@@ -812,7 +862,10 @@ export class DomStack {
     this.#templateDepMap = templateDepMap
     this.#pagesFileDepMap = pagesFileDepMap
     this.#globalDataDepPaths = globalDataDepPaths
+    this.#settingsDepPaths = settingsDepPaths
+    this.#dependencyAnalysisFailed = dependencyAnalysisFailed
     this.#esbuildEntryPoints = esbuildEntryPoints
+    this.#esbuildDepPaths = esbuildDepPaths
   }
 
   /**