feat: incremental builds#95
Conversation
🧭 Changeset detectedMerging this PR will release the following updates: maudit (Cargo) — minor version bumpMinor changes
Patch changes
maudit-cli (Cargo) — patch version bumpPatch changes
maudit-macros (Cargo) — minor version bumpMinor changes
|
✅ Deploy Preview for maudit-framework ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Merging this PR will improve performance by 65.41%
Performance Changes
Comparing Footnotes |
There was a problem hiding this comment.
Pull request overview
This PR introduces an incremental build system for Maudit that tracks per-page content and asset dependencies, enabling subsequent builds (and dev rebuilds) to skip re-rendering unchanged pages for faster feedback loops.
Changes:
- Add a persisted build cache and dependency-diffing logic (content entries, iterated sources, asset fingerprints, stale output cleanup).
- Introduce tracked content access APIs (
ctx.content::<T>(...)) to record dependencies duringrender()/pages(). - Improve dev workflow by detecting whether Rust changes require recompilation vs just rerunning the built binary; add e2e coverage for the behavior.
Reviewed changes
Copilot reviewed 55 out of 56 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| website/src/routes/news.rs | Update routes to use tracked ctx.content::<T>() and iterator API. |
| website/src/routes/docs.rs | Update docs routes to use tracked ctx.content::<T>(). |
| website/src/layout/docs_sidebars.rs | Update sidebar content iteration to use tracked entries iterator. |
| website/content/docs/library.md | Update docs snippets for new RouteAssets + DynamicRouteContext::new APIs. |
| website/content/docs/content.md | Update docs snippets for new tracked content APIs and new Entry::create signature. |
| examples/oubli-basics/src/routes/index.rs | Update example to tracked content access and iterator API. |
| examples/markdown-components/src/routes.rs | Update example to tracked content access API. |
| examples/library/src/routes/index.rs | Update example to tracked content access and iterator API. |
| examples/library/src/routes/article.rs | Update example to tracked content access API. |
| examples/library/src/build.rs | Update example for new RouteAssets::new and DynamicRouteContext::new. |
| examples/blog/src/routes/index.rs | Update example to tracked content access and iterator API. |
| examples/blog/src/routes/article.rs | Update example to tracked content access API. |
| e2e/tests/test-utils.ts | Add dev-server log capture utilities to support log-based polling in tests. |
| e2e/tests/hot-reload.spec.ts | Expand hot-reload coverage to assert recompile vs rerun behavior via logs. |
| e2e/fixtures/hot-reload/data.txt | Add fixture file for “non-Rust change triggers rerun” scenario. |
| crates/oubli/src/lib.rs | Update content entry creation to new dependency list parameter. |
| crates/oubli/src/archetypes/blog.rs | Update archetype helpers to tracked content access and iterator API. |
| crates/maudit/src/route.rs | Add content access logging + tracked content handles; optimize URL/path building; add incremental hooks. |
| crates/maudit/src/logging.rs | Adjust quiet-mode behavior and filter noisy module logs. |
| crates/maudit/src/content/tracked.rs | Introduce tracked content wrapper + access log for incremental dependency tracking. |
| crates/maudit/src/content/markdown/shortcodes_tests.rs | Update tests for new RouteAssets/PageContext fields/signatures. |
| crates/maudit/src/content/markdown.rs | Update markdown content to record file dependencies for incremental hashing. |
| crates/maudit/src/content.rs | Add dependency model, switch entries to map for faster lookup, extend internal content-source interface. |
| crates/maudit/src/build/options.rs | Add incremental/cache options and option hashing for cache invalidation. |
| crates/maudit/src/build/metadata.rs | Track cached vs rendered pages and expose “has_changes” utility. |
| crates/maudit/src/build/cache.rs | Add persisted incremental build cache format and diffing logic. |
| crates/maudit/src/build.rs | Integrate incremental cache into build pipeline; add cache hits, stale cleanup, bundling skip logic. |
| crates/maudit/src/assets/image_cache.rs | Persist image cache separately; add invalidation via mtime/size and GC support. |
| crates/maudit/src/assets.rs | Add shared asset-hash cache to reduce redundant hashing across pages. |
| crates/maudit/Cargo.toml | Add dependencies for cache persistence/tests (bincode, serde derive, serial_test). |
| crates/maudit-macros/src/lib.rs | Extend route macro to support always_revalidate and pass source-entry hints through pages. |
| crates/maudit-cli/src/dev/server.rs | Add StatusManager wrapper for status broadcasting + persistent state. |
| crates/maudit-cli/src/dev/dep_tracker.rs | Add .d-file dependency tracker to decide recompile vs rerun. |
| crates/maudit-cli/src/dev/build.rs | Add rerun path and dependency tracking; refactor status handling via StatusManager. |
| crates/maudit-cli/src/dev.rs | Use dependency tracker decision to rerun vs recompile on file changes. |
| crates/maudit-cli/Cargo.toml | Add depinfo + tempfile test dependency. |
| benchmarks/realistic-blog/src/routes/index.rs | Update benchmark routes to tracked content APIs. |
| benchmarks/realistic-blog/src/routes/article.rs | Update benchmark routes to tracked content APIs. |
| benchmarks/realistic-blog/src/lib.rs | Disable incremental builds for baseline benchmark behavior. |
| benchmarks/realistic-blog/benches/build.rs | Ensure cache dir is cleaned for benchmark repeatability. |
| benchmarks/overhead/src/lib.rs | Disable incremental builds for baseline benchmark behavior. |
| benchmarks/overhead/benches/build.rs | Ensure cache dir is cleaned for benchmark repeatability. |
| benchmarks/md-benchmark/src/page.rs | Update benchmark to tracked content APIs. |
| benchmarks/md-benchmark/src/lib.rs | Disable incremental builds for baseline benchmark behavior. |
| benchmarks/md-benchmark/benches/build.rs | Ensure cache dir is cleaned for benchmark repeatability. |
| benchmarks/incremental/src/page.rs | Add new incremental-specific benchmark route. |
| benchmarks/incremental/src/main.rs | Add binary entrypoint for incremental benchmark crate. |
| benchmarks/incremental/src/lib.rs | Add incremental benchmark harness using incremental builds. |
| benchmarks/incremental/benches/build.rs | Add benchmark for incremental “no changes” rebuilds. |
| benchmarks/incremental/Cargo.toml | Add new benchmark crate manifest. |
| Cargo.lock | Lockfile updates for new Rust dependencies. |
| .sampo/config.toml | Update Sampo ignore list. |
| .sampo/changesets/stalwart-stormcaller-vipunen.md | Changeset: dynamic route perf note. |
| .sampo/changesets/somber-warden-aurelien.md | Changeset: incremental builds release note. |
| .sampo/changesets/cranky-sage-tursas.md | Changeset: CLI rerun optimization release note. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| pub fn needs_recompile(&self, changed_paths: &[PathBuf]) -> bool { | ||
| for changed_path in changed_paths { | ||
| // Normalize the changed path to handle relative vs absolute paths | ||
| let changed_path_canonical = changed_path.canonicalize().ok(); | ||
|
|
||
| for (dep_path, last_modified) in &self.dependencies { | ||
| // Try to match both exact path and canonical path | ||
| let matches = changed_path == dep_path | ||
| || changed_path_canonical.as_ref() == Some(dep_path) | ||
| || dep_path.canonicalize().ok().as_ref() == changed_path_canonical.as_ref(); | ||
|
|
| pub fn save(&self, cache_dir: &Path) -> std::io::Result<()> { | ||
| fs::create_dir_all(cache_dir)?; | ||
| let path = cache_dir.join(BUILD_CACHE_FILENAME); | ||
| let tmp_path = cache_dir.join(format!("{}.tmp", BUILD_CACHE_FILENAME)); | ||
| let bytes = bincode::serialize(self).expect("BuildCache serialization should not fail"); | ||
| fs::write(&tmp_path, bytes)?; | ||
| fs::rename(&tmp_path, &path)?; | ||
| Ok(()) |
| Some(manifest) | ||
| let bytes = | ||
| bincode::serialize(&persisted).expect("ImageCache serialization should not fail"); | ||
| fs::write(&tmp_path, bytes)?; |
| function appendLines(data: Buffer) { | ||
| const lines = data.toString().split("\n"); | ||
| for (const line of lines) { | ||
| if (line.trim() !== "") { | ||
| capturedLogs.push(line); | ||
| } | ||
| } | ||
| } |
| cargo/maudit-cli: patch | ||
| --- | ||
|
|
||
| The Maudit CLI will now directly rerun the website's binary instead of using Cargo when changes do notrequire recompilation, this on average speeds up the feedback loop by 300-1000ms. |
| } | ||
|
|
||
| // Acquire semaphore to ensure only one build/run happens at a time | ||
| let _ = self.build_semaphore.acquire().await?; |
| @@ -61,13 +251,9 @@ impl BuildManager { | |||
| let _ = self.build_semaphore.acquire().await?; | |||
| fn render(&self, ctx: &mut PageContext) -> impl Into<RenderResult> { | ||
| let entry = ctx | ||
| .content | ||
| .get_source::<MyType>("my_data") | ||
| .get_entry("0"); | ||
| let source = ctx.collection::<MyType>("my_data"); | ||
| let entry = source.get_entry("0"); |
| @@ -348,27 +358,26 @@ impl<T> ContentSource<T> { | |||
| { | |||
| Self { | |||
| name: name.into(), | |||
| entries: vec![], | |||
| entries: FxHashMap::default(), | |||
| init_method: entries, | |||
| } | |||
| } | |||
|
|
|||
| pub fn get_entry(&self, id: &str) -> &Entry<T> { | |||
| self.entries | |||
| .iter() | |||
| .find(|entry| entry.id == id) | |||
| .get(id) | |||
| .unwrap_or_else(|| panic!("Entry with id '{}' not found", id)) | |||
| } | |||
|
|
|||
| pub fn get_entry_safe(&self, id: &str) -> Option<&Entry<T>> { | |||
| self.entries.iter().find(|entry| entry.id == id) | |||
| self.entries.get(id) | |||
| } | |||
|
|
|||
| pub fn into_params<P>(&self, cb: impl FnMut(&Entry<T>) -> P) -> Vec<P> | |||
| where | |||
| P: Into<PageParams>, | |||
| { | |||
| self.entries.iter().map(cb).collect() | |||
| self.entries.values().map(cb).collect() | |||
| } | |||
|
|
|||
| pub fn into_pages<Params, Props>( | |||
| @@ -378,7 +387,7 @@ impl<T> ContentSource<T> { | |||
| where | |||
| Params: Into<PageParams>, | |||
| { | |||
| self.entries.iter().map(cb).collect() | |||
| self.entries.values().map(cb).collect() | |||
| } | |||
Fixes #
What does this change?
How is it tested?
How is it documented?