Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
154 lines (101 loc) · 11.2 KB

ARCHITECTURE.md

File metadata and controls

154 lines (101 loc) · 11.2 KB
Raw

Architecture

An overview of how the docs site is built. For writing guidelines see CONTRIBUTING.md; for CI and deploys see WORKFLOWS.md.

Overview

This is a Docusaurus project, deployed on Cloudflare Pages. The Docusaurus configuration is in docusaurus.config.js. Search is provided by Algolia DocSearch.

Pages are MDX under the hood, but mostly carry .md extensions for Docusaurus legacy reasons.

Key architectural files and folders:

Path Contents
docs/ Main documentation, rendered at /docs/*
tutorials/ Tutorial pages, rendered at /tutorials/* via a second plugin instance
src/components/ Custom React components used inside MDX
src/theme/ Swizzled Docusaurus theme components
src/pages/ Custom routes outside docs/tutorials (homepage, style guide, license pages)
plugins/ Custom local Docusaurus plugins
static/ Files served at the site root (fonts, images, downloadable notebooks)
worker/ Cloudflare Worker source: redirects + server-side tracking
utils/ Scripts for refreshing dbt reference data
sidebars.js Sidebar transformation logic
src/componentVersions.js Source of truth for component version strings

Gotchas

Non-obvious things that could cause confusion if you don't know about them:

  • static/_redirects is no longer active. Legacy Netlify file kept for reference only. Live redirects are in worker/redirects.js. The yarn build:cf script deletes the auto-generated build/_redirects so nothing leaks.
  • yarn build is the only link checker. yarn start doesn't catch broken internal links or missing anchors. The build is configured with onBrokenLinks: 'throw' + onBrokenAnchors: 'throw'.
  • componentVersions.js has no JS imports. It's referenced from MDX pages only — grep versions. to find usage.
  • Tailwind and Infima coexist deliberately but unevenly. See Styling and CSS.
  • The sidebar doesn't match the folder tree. sidebars.js reshapes the autogenerated tree before render: it groups top-level folders into named sections (sidebar_custom_props.header), hoists two sections' children up a level (Signals, Components), and swaps type: link docs for external-link items.
  • Tutorials are a separate plugin instance and don't share docs conventions — different sidebar, different link rules. See Tutorials.
  • The Cloudflare Worker tracks page-view events server-side to track LLM crawlers.

Cloudflare Worker

worker/index.js runs on every request and does three things:

  1. Server-side Snowplow tracking
  2. Forced redirects via findForcedRedirect(pathname), checked before asset fetch, returns 301 on match
  3. Fallback redirects via findFallbackRedirect(pathname), checked only after a 404

Both redirect tiers are defined in worker/redirects.js. move.sh appends to it automatically.

Custom plugins

Live under plugins/.

LLMs.txt and Markdown generation

This plugin generates LLM artifacts during build:

  • A .md file for each page
  • An llms.txt index containing the page title, description, and link to the markdown version
  • A concatenated llms-full.txt

The llms.txt index includes files in both docs/ and tutorials/. It excludes pages with type: link and routes in the excludeRoutes array. Files marked as sidebar_custom_props.legacy or similar are indexed, but are suffixed with [previous version].

The llms-full.txt file concatenates all the per-page markdown files, except for the [previous version] pages.

The plugin includes scripts for converting different types of content or components into Markdown.

JSON-LD schema

This plugin adds structured JSON-LD schema metadata to the <head> of each page. This metadata helps with SEO and AIO/GEO/LLM consumption.

The schema itself is generated by the SchemaPlugin component. All pages have type TechArticle.

The plugin uses the title, description, and keywords from the page frontmatter. There's a CI workflow to enforce these fields for this reason.

Cookies and tracking

The site uses cookie-though for consent management. It's initialized in cookieConsent.js.

User preferences are stored in a cookie keyed by COOKIE_PREF_KEY, with per-policy flags (analytics:0 / analytics:1).

The user-facing preferences page is src/pages/cookie-preferences.mdx, linked to from the site footer.

Tracking scripts

The repo has multiple tracking implementations. Each tracking script manages its own consent behavior independently.

Script Purpose Default behavior Consent opt-in Consent opt-out
snowplow.js Snowplow browser tracker Anonymous tracking Calls disableAnonymousTracking(), sets cookies, and fires a new page view Removes _sp5_* and sp cookies and reloads (_sp_biz1_* cookies aren't removed on opt-out)
google.js Google Analytics / Google tag No scripts loaded Starts tracking, sets cookies Removes _ga, _gid, and _gat_* cookies and reloads
reoTracking.js Reo.dev tracker Loaded unconditionally N/A N/A
src/qualified.js Qualified chat/conversion tracking Loaded unconditionally N/A N/A
worker/index.js Page view tracking for .md and llms.txt requests Anonymous tracking N/A N/A

Styling and CSS

Most custom CSS lives in src/css/custom.css. The file is imported in docusaurus.config.js and applies globally. Some components also have their own CSS modules, e.g. src/theme/DocItem/Layout/styles.module.css.

Code blocks use custom Prism syntax highlighting themes: src/theme/PrismThemes/snowplow-light.js and src/theme/PrismThemes/snowplow-dark.js.

Tailwind and Infima

Docusaurus uses Infima as its native CSS framework. This provides basic styling and layout, and is used by the default Docusaurus components. This repo also uses Tailwind CSS for custom styles and components.

Tailwind runs with important: true and preflight: false in tailwind.config.js, so it can override Infima without resetting it.

Practical consequence: CSS output might not render as expected, or might differ between local and production environments. If you see unexpected styling, it's likely a Tailwind vs Infima issue.

Mermaid

The Mermaid theme component renders diagrams as SVG. It's been swizzled to add the source Mermaid code in the DOM, alongside the rendered SVG, for ingestion by LLMs.

There's also a module src/js/mermaidEnlarge.js that adds click-to-enlarge functionality on mobile, since small SVGs can be hard to read.

Tutorials

Served at /tutorials/* via a second @docusaurus/plugin-content-docs instance configured separately.

The plugin has no sidebarPath, so navigation is custom and not file-tree-derived. The tutorials use the components in src/components/tutorials.

Internal link conventions inside tutorials/ differ from docs/ — see tutorials/_README.md. Downloadable notebooks live as static assets in static/notebooks/.

LLM support

As well as the custom llms.txt and JSON-LD schema plugins, the docs have additional features designed to make them easier for LLMs to parse and understand:

  • CLAUDE.md instructions
  • LLM-targeted style guide at src/pages/style-guide/llm/index.md.
  • Download or Copy Markdown buttons on every docs page, implemented in src/theme/DocItem/Layout/index.tsx. Note that these buttons don't provide the correct output when running locally.

Product Fruits feedback widget

The site uses Product Fruits for in-product feedback collection. It reads user/session IDs from the Snowplow _sp_biz1_id cookie to link feedback to analytics data.

The ProductFruits widget is initialized in src/theme/Root.js using the production Product Fruits workspace code.

Snowplow event-sending component

The Get Started > Tracking page docs/get-started/tracking/index.md includes a "send test events" widget that allows users to send pre-defined test events to their Snowplow pipeline, directly from the docs.

This is implemented in src/components/FirstSteps/index.tsx. It uses a sandboxed Snowplow tracker from static/js/sandboxed-sp.js to keep test events isolated from the site's own tracking.