feat: migrate Phase 2 SSR - replace DustJS/jQuery/Sammy with Nunjucks/Vanilla JS#361
feat: migrate Phase 2 SSR - replace DustJS/jQuery/Sammy with Nunjucks/Vanilla JS#361mikaello wants to merge 8 commits into
Conversation
…/Vanilla JS Replace client-side DustJS rendering with server-side Nunjucks rendering. - Add src/schema_parser.js: server-side port of public/js/schema.js - Add src/top_level.njk and templates/*.njk: Nunjucks templates - Rewrite src/static_content.js: uses Nunjucks + buildAvroDocContext - Rewrite public/js/avrodoc.js: Vanilla JS hash routing + popovers + search - Update app.js: load JSON schemata at startup, remove dust-templates route - Remove DustJS, jQuery, Sammy.js, markdown.js from vendor and dependencies - Remove public/js/schema.js (logic moved to src/schema_parser.js) - Update eslint.config.js: remove jQuery globals - Update test assertion to check for SSR output marker (data-route) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
e574531 to
a725934
Compare
Avrodoc Preview 🔍A live preview of the generated documentation is available at: This preview is built from the |
Performance note: scaling with large schema setsThe SSR approach renders all content upfront. Per-type cost is ~3.8 KB (2.6 KB section HTML + 1.2 KB popover JSON). At 1000 types this is ~4 MB total HTML; at 5000 types ~19 MB. The old lazy approach (client-side dust rendering) also scaled poorly (blocking JS parse + large embedded JSON), but the trade-off shifts at large scale. Two concrete optimizations for a future iteration1. Eliminate popover data duplication (~50% size reduction per type) The popover JSON (in // In setupPopovers(), replace JSON lookup with DOM lookup:
var section = findSection('#/schema/' + encodeURIComponent(filename) + '/' + encodeURIComponent(qualifiedName));
var content = section ? section.innerHTML : null;Remove 2. Lazy section rendering for server mode In |
|
I tested this rewrite against the avrodoc-plus-demo stress-test repo (497 schemas covering full Avro 1.12.1 spec). See the test PR here: mikaello/avrodoc-plus-demo#2 ResultThe build fails with the same cross-file reference error (#362): What changed vs. the originalThe SSR rewrite improves the failure mode — it's now a hard build-time crash instead of a silent browser console error. That's a meaningful step forward. However, Suggested fixA first-pass pre-registration step in // Pass 1: register all top-level named type names into shared_types
// (without resolving field types)
for (const { filename, json } of input_schemata) {
preRegisterTopLevelTypes(json, shared_types);
}
// Pass 2: full parse (existing loop) — cross-file refs now resolvable
for (const { filename, json } of input_schemata) {
// ... current parseAvroSchema call
}
|
Instead of pre-rendering named_type_details HTML twice (once for the section and once for the popover JSON blob), popovers now read their content directly from the already-rendered section elements in the DOM. Changes: - Remove buildPopoverData() from static_content.js - Remove <script id="popover-data"> JSON blob from top_level.njk - Remove popover_title.njk (was only used by buildPopoverData) - Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM queries for popover title (h2.namespace / h1.type-name) and content (.type-details innerHTML) - Wrap named_type.njk body in <div class="type-details"> so popovers can extract content without the heading elements This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction of the per-type overhead) and eliminates double-rendering at startup. Fixes: #361 (comment)
Instead of pre-rendering named_type_details HTML twice (once for the section and once for the popover JSON blob), popovers now read their content directly from the already-rendered section elements in the DOM. Changes: - Remove buildPopoverData() from static_content.js - Remove <script id="popover-data"> JSON blob from top_level.njk - Remove popover_title.njk (was only used by buildPopoverData) - Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM queries for popover title (h2.namespace / h1.type-name) and content (.type-details innerHTML) - Wrap named_type.njk body in <div class="type-details"> so popovers can extract content without the heading elements This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction of the per-type overhead) and eliminates double-rendering at startup. Fixes: #361 (comment)
Instead of pre-rendering named_type_details HTML twice (once for the section and once for the popover JSON blob), popovers now read their content directly from the already-rendered section elements in the DOM. Changes: - Remove buildPopoverData() from static_content.js - Remove <script id="popover-data"> JSON blob from top_level.njk - Remove popover_title.njk (was only used by buildPopoverData) - Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM queries for popover title (h2.namespace / h1.type-name) and content (.type-details innerHTML) - Wrap named_type.njk body in <div class="type-details"> so popovers can extract content without the heading elements This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction of the per-type overhead) and eliminates double-rendering at startup. Fixes: #361 (comment)
…rder (#363) * fix: resolve cross-file type references regardless of filename sort order When multiple .avsc files are processed, schemas referencing named types (record/enum/fixed) defined in other files would fail with "Unknown type name" if the defining file sorted alphabetically after the referencing file. Root cause: buildAvroDocContext processed schemata in input order (reflecting filename sort), so type references were resolved eagerly before definitions from later files were registered in shared_types. Fix: topologically sort schemata by dependency order before parsing. A lightweight pre-scan (extractTypeInfo) collects each schema's defined and referenced type names; Kahn's algorithm then orders schemata so definitions always precede their references. Falls back to original order if a cycle is detected. Fixes #362 * fix: remove redundant JSON.parse on already-parsed schema input parseAvroSchema received raw schema_json that had already been parsed by readJSON. The JSON.parse guard would throw a SyntaxError for any schema file whose content is a bare primitive type string (e.g. "boolean"), which is valid per the Avro spec. Remove the typeof-string guard entirely — callers always pass pre-parsed values. Add a test and fixture for the bare-primitive case. Reported in #363 review.
c222236 to
926568b
Compare
Instead of pre-rendering named_type_details HTML twice (once for the section and once for the popover JSON blob), popovers now read their content directly from the already-rendered section elements in the DOM. Changes: - Remove buildPopoverData() from static_content.js - Remove <script id="popover-data"> JSON blob from top_level.njk - Remove popover_title.njk (was only used by buildPopoverData) - Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM queries for popover title (h2.namespace / h1.type-name) and content (.type-details innerHTML) - Wrap named_type.njk body in <div class="type-details"> so popovers can extract content without the heading elements This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction of the per-type overhead) and eliminates double-rendering at startup. Fixes: #361 (comment)
…#367) Two UX fixes: 1. Popover persists after clicking a link Dismiss the active popover immediately on every hashchange, so it never carries over to the next page regardless of how navigation was triggered (click, keyboard, sidebar). 2. Scroll position not restored on browser back button Save the scroll offset for each visited hash in `scrollPositions`. A `popstate` flag distinguishes back/forward navigation from a forward link click. On back/forward, the saved offset is restored; on forward navigation the page still scrolls to the top.
Instead of pre-rendering named_type_details HTML twice (once for the section and once for the popover JSON blob), popovers now read their content directly from the already-rendered section elements in the DOM. Changes: - Remove buildPopoverData() from static_content.js - Remove <script id="popover-data"> JSON blob from top_level.njk - Remove popover_title.njk (was only used by buildPopoverData) - Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM queries for popover title (h2.namespace / h1.type-name) and content (.type-details innerHTML) - Wrap named_type.njk body in <div class="type-details"> so popovers can extract content without the heading elements This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction of the per-type overhead) and eliminates double-rendering at startup. Fixes: #361 (comment)
Phase 2: Server-Side Rendering Migration
Replaces client-side DustJS rendering + jQuery + Sammy.js with server-side Nunjucks rendering + Vanilla JS.
Changes
New files:
src/schema_parser.js— server-side port ofpublic/js/schema.js+buildAvroDocContext()src/top_level.njk— top-level Nunjucks template (replacessrc/top_level.dust)templates/*.njk— Nunjucks templates for all schema viewsModified files:
src/static_content.js— fully rewritten using Nunjucks SSR; all sections pre-rendered server-sidepublic/js/avrodoc.js— replaced with Vanilla JS (hash routing, popovers, search)app.js— loads JSON schemata at startup, removed dust-templates routeeslint.config.js— removed jQuery/$ globalssrc/avrodoc.test.js— updated assertion to check for SSR output marker (data-route)Removed:
public/js/schema.js, all vendor DustJS/jQuery/Sammy/markdown files,templates/*.dust, dustjs npm dependenciesArchitecture
All HTML is rendered server-side. The browser receives
<section data-route="..."elements and a<script id="popover-data"JSON blob. Vanilla JS handles hash routing, Bootstrap popovers, and sidebar search.