Skip to content

feat: migrate Phase 2 SSR - replace DustJS/jQuery/Sammy with Nunjucks/Vanilla JS#361

Open
mikaello wants to merge 8 commits into
mainfrom
migrate-phase2-ssr
Open

feat: migrate Phase 2 SSR - replace DustJS/jQuery/Sammy with Nunjucks/Vanilla JS#361
mikaello wants to merge 8 commits into
mainfrom
migrate-phase2-ssr

Conversation

@mikaello

Copy link
Copy Markdown
Owner

Phase 2: Server-Side Rendering Migration

Replaces client-side DustJS rendering + jQuery + Sammy.js with server-side Nunjucks rendering + Vanilla JS.

Changes

New files:

  • src/schema_parser.js — server-side port of public/js/schema.js + buildAvroDocContext()
  • src/top_level.njk — top-level Nunjucks template (replaces src/top_level.dust)
  • templates/*.njk — Nunjucks templates for all schema views

Modified files:

  • src/static_content.js — fully rewritten using Nunjucks SSR; all sections pre-rendered server-side
  • public/js/avrodoc.js — replaced with Vanilla JS (hash routing, popovers, search)
  • app.js — loads JSON schemata at startup, removed dust-templates route
  • eslint.config.js — removed jQuery/$ globals
  • src/avrodoc.test.js — updated assertion to check for SSR output marker (data-route)

Removed: public/js/schema.js, all vendor DustJS/jQuery/Sammy/markdown files, templates/*.dust, dustjs npm dependencies

Architecture

All HTML is rendered server-side. The browser receives <section data-route="..." elements and a <script id="popover-data" JSON blob. Vanilla JS handles hash routing, Bootstrap popovers, and sidebar search.

mikaello and others added 2 commits April 12, 2026 22:14
…/Vanilla JS

Replace client-side DustJS rendering with server-side Nunjucks rendering.
- Add src/schema_parser.js: server-side port of public/js/schema.js
- Add src/top_level.njk and templates/*.njk: Nunjucks templates
- Rewrite src/static_content.js: uses Nunjucks + buildAvroDocContext
- Rewrite public/js/avrodoc.js: Vanilla JS hash routing + popovers + search
- Update app.js: load JSON schemata at startup, remove dust-templates route
- Remove DustJS, jQuery, Sammy.js, markdown.js from vendor and dependencies
- Remove public/js/schema.js (logic moved to src/schema_parser.js)
- Update eslint.config.js: remove jQuery globals
- Update test assertion to check for SSR output marker (data-route)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mikaello mikaello force-pushed the migrate-phase2-ssr branch from e574531 to a725934 Compare April 12, 2026 20:14
@github-actions

github-actions Bot commented Apr 12, 2026

Copy link
Copy Markdown

Avrodoc Preview 🔍

A live preview of the generated documentation is available at:
👉 https://mikaello.github.io/avrodoc-plus/pr-preview/pr-361/

This preview is built from the schemata/ directory and updates on every push to this PR.

@mikaello

Copy link
Copy Markdown
Owner Author

Performance note: scaling with large schema sets

The SSR approach renders all content upfront. Per-type cost is ~3.8 KB (2.6 KB section HTML + 1.2 KB popover JSON). At 1000 types this is ~4 MB total HTML; at 5000 types ~19 MB. The old lazy approach (client-side dust rendering) also scaled poorly (blocking JS parse + large embedded JSON), but the trade-off shifts at large scale.

Two concrete optimizations for a future iteration

1. Eliminate popover data duplication (~50% size reduction per type)

The popover JSON (in <script id="popover-data" type="application/json">) contains named_type_details HTML that is already present in the hidden <section> elements. Instead, popovers can read their content directly from the DOM:

// In setupPopovers(), replace JSON lookup with DOM lookup:
var section = findSection('#/schema/' + encodeURIComponent(filename) + '/' + encodeURIComponent(qualifiedName));
var content = section ? section.innerHTML : null;

Remove buildPopoverData() in static_content.js, and the popover-only templates (popover_title.njk, named_type_details.njk). This cuts per-type cost from ~3.8 KB to ~2.6 KB and eliminates double-rendering of named_type_details at server startup.

2. Lazy section rendering for server mode

In app.js (server mode, not CLI inline mode), pre-render only the sidebar and the #/ default section. Add an endpoint GET /section/:filename/:qualifiedName that renders and returns a single section on demand. The client fetches and injects it on first navigation, caching it in the DOM. This keeps initial HTML small (~50 KB) and scales to arbitrarily large schema sets without changing the CLI inline behaviour.

@mikaello

Copy link
Copy Markdown
Owner Author

I tested this rewrite against the avrodoc-plus-demo stress-test repo (497 schemas covering full Avro 1.12.1 spec). See the test PR here: mikaello/avrodoc-plus-demo#2

Result

The build fails with the same cross-file reference error (#362):

Unknown type name "com.example.enums.LogLevel" at logLevel

What changed vs. the original

The SSR rewrite improves the failure mode — it's now a hard build-time crash instead of a silent browser console error. That's a meaningful step forward.

However, schema_parser.jsbuildAvroDocContext uses the same eager type resolution as the original public/js/schema.js. Schemas are processed in alphabetical filename order; if schema A (complex/) references a type defined in schema B (enums/), and B sorts after A, the lookup fails.

Suggested fix

A first-pass pre-registration step in buildAvroDocContext before the main parse loop would fix it:

// Pass 1: register all top-level named type names into shared_types
// (without resolving field types)
for (const { filename, json } of input_schemata) {
  preRegisterTopLevelTypes(json, shared_types);
}

// Pass 2: full parse (existing loop) — cross-file refs now resolvable
for (const { filename, json } of input_schemata) {
  // ... current parseAvroSchema call
}

preRegisterTopLevelTypes would only need to walk the top-level schema and nested types arrays (for protocols) to collect record/enum/fixed fullnames — no field resolution needed.

mikaello added a commit that referenced this pull request Apr 13, 2026
Instead of pre-rendering named_type_details HTML twice (once for the
section and once for the popover JSON blob), popovers now read their
content directly from the already-rendered section elements in the DOM.

Changes:
- Remove buildPopoverData() from static_content.js
- Remove <script id="popover-data"> JSON blob from top_level.njk
- Remove popover_title.njk (was only used by buildPopoverData)
- Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM
  queries for popover title (h2.namespace / h1.type-name) and content
  (.type-details innerHTML)
- Wrap named_type.njk body in <div class="type-details"> so popovers
  can extract content without the heading elements

This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction
of the per-type overhead) and eliminates double-rendering at startup.
Fixes: #361 (comment)
mikaello added a commit that referenced this pull request Apr 13, 2026
Instead of pre-rendering named_type_details HTML twice (once for the
section and once for the popover JSON blob), popovers now read their
content directly from the already-rendered section elements in the DOM.

Changes:
- Remove buildPopoverData() from static_content.js
- Remove <script id="popover-data"> JSON blob from top_level.njk
- Remove popover_title.njk (was only used by buildPopoverData)
- Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM
  queries for popover title (h2.namespace / h1.type-name) and content
  (.type-details innerHTML)
- Wrap named_type.njk body in <div class="type-details"> so popovers
  can extract content without the heading elements

This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction
of the per-type overhead) and eliminates double-rendering at startup.
Fixes: #361 (comment)
mikaello added a commit that referenced this pull request Apr 14, 2026
Instead of pre-rendering named_type_details HTML twice (once for the
section and once for the popover JSON blob), popovers now read their
content directly from the already-rendered section elements in the DOM.

Changes:
- Remove buildPopoverData() from static_content.js
- Remove <script id="popover-data"> JSON blob from top_level.njk
- Remove popover_title.njk (was only used by buildPopoverData)
- Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM
  queries for popover title (h2.namespace / h1.type-name) and content
  (.type-details innerHTML)
- Wrap named_type.njk body in <div class="type-details"> so popovers
  can extract content without the heading elements

This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction
of the per-type overhead) and eliminates double-rendering at startup.
Fixes: #361 (comment)
…rder (#363)

* fix: resolve cross-file type references regardless of filename sort order

When multiple .avsc files are processed, schemas referencing named types
(record/enum/fixed) defined in other files would fail with
"Unknown type name" if the defining file sorted alphabetically after
the referencing file.

Root cause: buildAvroDocContext processed schemata in input order
(reflecting filename sort), so type references were resolved eagerly
before definitions from later files were registered in shared_types.

Fix: topologically sort schemata by dependency order before parsing.
A lightweight pre-scan (extractTypeInfo) collects each schema's
defined and referenced type names; Kahn's algorithm then orders
schemata so definitions always precede their references. Falls back
to original order if a cycle is detected.

Fixes #362

* fix: remove redundant JSON.parse on already-parsed schema input

parseAvroSchema received raw schema_json that had already been parsed
by readJSON. The JSON.parse guard would throw a SyntaxError for any
schema file whose content is a bare primitive type string (e.g.
"boolean"), which is valid per the Avro spec.

Remove the typeof-string guard entirely — callers always pass
pre-parsed values. Add a test and fixture for the bare-primitive case.

Reported in #363 review.
@mikaello mikaello force-pushed the migrate-phase2-ssr branch from c222236 to 926568b Compare April 14, 2026 19:59
mikaello added a commit that referenced this pull request Apr 14, 2026
Instead of pre-rendering named_type_details HTML twice (once for the
section and once for the popover JSON blob), popovers now read their
content directly from the already-rendered section elements in the DOM.

Changes:
- Remove buildPopoverData() from static_content.js
- Remove <script id="popover-data"> JSON blob from top_level.njk
- Remove popover_title.njk (was only used by buildPopoverData)
- Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM
  queries for popover title (h2.namespace / h1.type-name) and content
  (.type-details innerHTML)
- Wrap named_type.njk body in <div class="type-details"> so popovers
  can extract content without the heading elements

This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction
of the per-type overhead) and eliminates double-rendering at startup.
Fixes: #361 (comment)
…#367)

Two UX fixes:

1. Popover persists after clicking a link
   Dismiss the active popover immediately on every hashchange, so it
   never carries over to the next page regardless of how navigation
   was triggered (click, keyboard, sidebar).

2. Scroll position not restored on browser back button
   Save the scroll offset for each visited hash in `scrollPositions`.
   A `popstate` flag distinguishes back/forward navigation from a
   forward link click. On back/forward, the saved offset is restored;
   on forward navigation the page still scrolls to the top.
mikaello added a commit that referenced this pull request Apr 14, 2026
Instead of pre-rendering named_type_details HTML twice (once for the
section and once for the popover JSON blob), popovers now read their
content directly from the already-rendered section elements in the DOM.

Changes:
- Remove buildPopoverData() from static_content.js
- Remove <script id="popover-data"> JSON blob from top_level.njk
- Remove popover_title.njk (was only used by buildPopoverData)
- Rewrite setupPopovers() in avrodoc.js to use findSection() + DOM
  queries for popover title (h2.namespace / h1.type-name) and content
  (.type-details innerHTML)
- Wrap named_type.njk body in <div class="type-details"> so popovers
  can extract content without the heading elements

This cuts per-type HTML cost from ~3.8 KB to ~2.6 KB (~50% reduction
of the per-type overhead) and eliminates double-rendering at startup.
Fixes: #361 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant