Skip to content

feat: create schema.org vocabulary package - #3

Merged
jonathanprozzi merged 2 commits into
mainfrom
feat/schema-org-vocabulary-package
Jun 12, 2026
Merged

feat: create schema.org vocabulary package#3
jonathanprozzi merged 2 commits into
mainfrom
feat/schema-org-vocabulary-package

Conversation

@jonathanprozzi

@jonathanprozzi jonathanprozzi commented Jun 1, 2026

Copy link
Copy Markdown
Member

Summary

Creates @0xintuition/schema-org as a standalone public alpha package for the pinned schema.org vocabulary foundation layer.

What changed:

  • Added packages/schema-org with package manifest, README, TypeScript configs, tests, and ESM source entrypoints.
  • Vendored the pinned schema.org V30.0 JSON-LD snapshot at packages/schema-org/vendor/schemaorg-v30.0.jsonld.
  • Generated one typed TypeScript artifact per schema.org schema: class under src/generated/types/<Type>.ts.
  • Added src/generated/index.ts aggregation plus runtime helpers: getType, getPropertiesFor, hasType, listTypes, and normalizeSchemaOrgName.
  • Added reproducible generation scripts: schema-org:generate, schema-org:refresh, and schema-org:check-generated.
  • Added CI coverage for schema-org:check-generated.
  • Added schema-org to root dry-run and clean-room tarball smoke coverage.

Approach

The source vocabulary is pinned to schema.org V30.0:

https://schema.org/version/30.0/schemaorg-current-https.jsonld

Generation reads the vendored V30.0 snapshot by default so local generation and CI do not depend on a live network fetch. Maintainers can explicitly refresh the vendored snapshot with bun run schema-org:refresh.

The generator parses the flat @graph, separates rdfs:Class from rdf:Property, filters to schema.org schema: classes/properties, and inverts property schema:domainIncludes onto each owning class. Each type stores only directly declared properties plus its resolved subClassOf chain. getPropertiesFor(name) derives inherited properties at runtime and tags each with originType / originTypeId.

Example: normalized storage vs resolved properties

Generated Book stores only Book-owned properties:

  • subClassOf: ['CreativeWork', 'Thing']
  • own properties include isbn, bookEdition, and numberOfPages
  • inherited properties like name and author are not copied into Book.ts

At runtime, getPropertiesFor('Book') walks Book -> CreativeWork -> Thing, dedupes by property id, and tags each property with provenance:

  • name has originType: 'Thing'
  • author has originType: 'CreativeWork'
  • isbn has originType: 'Book'

This keeps generated artifacts normalized while still exposing the resolved inherited view consumers need.

V30.0 generated:

  • 933 schema: type modules
  • 1,521 schema: property records

Review follow-up

  • Made generation hermetic by default through the vendored V30.0 snapshot and explicit --refresh path.
  • Added the schema.org generated drift check to CI.
  • Changed subclass chain resolution to nearest-parent-first BFS ordering for multi-inheritance cases.
  • Expanded tests for inherited properties, multi-inheritance, data types, identifier-unsafe names such as 3DModel, and schema: / full URL lookup.
  • Documented safe export access for schema.org names that are not safe exact JavaScript identifiers, including 3DModel, Number, Date, and Map.

Open decisions flagged

  • Package name remains provisional as @0xintuition/schema-org.
  • Artifact entrypoint is typed .ts consts only; no raw .json files were added.
  • No dependency from classifications or predicates is introduced in this PR.
  • Flattened all-properties-per-type artifacts remain deferred; inherited properties are derived through the accessor.

Validation

Current follow-up validation:

  • bun run check
  • bun run schema-org:check-generated
  • cd packages/schema-org && bun run check
  • cd packages/schema-org && bun run test
  • cd packages/schema-org && bun run typecheck
  • cd packages/schema-org && bun run build

Initial package validation also covered root build/typecheck/test, bun run pack:dry-run, and bun run smoke:tarballs. Root tests pass; the existing React test suite still emits act(...) warnings during passing tests.

@linear

linear Bot commented Jun 2, 2026

Copy link
Copy Markdown
ENG-12255 Create schema.org JSON package

Summary

Create @0xintuition/schema-org — a package that codegens the complete schema.org vocabulary (pinned to V30.0) into per-type JSON as an immutable foundation layer. Classifications and predicates reference it as the canonical superset of entity types + properties; our hand-picked fields become a validated subset of this complete set.

Why

schema.org is the "complete menu" of entity types + properties. Today's classifications specs hand-pick a handful of fields per entity (opinion-based first pass — the ~40 we have were an initial stab). To deliver the "meta-language for building atoms" — telling builders which fields are atom-data (handled well by backend) vs. metadata-via-triples — we need the full set codified so we can:

  • show every available field, with provenance (name is from Thing, author from CreativeWork);
  • mark Intuition's recommended subset on top — superset architecture, NOT an extension ("we point out which fields Intuition cares about");
  • drive UI: atom-creation forms (classifications) + claim-predicate dropdowns (predicates).

Locked in the 2026-06-01 jam (JP + Kames, w/ Billy). This is the foundation that unblocks ENG-12256 (predicates backfill) and ENG-12257 (classifications metadataPredicates).

Confirmed data source

  • Canonical dump: https://schema.org/version/latest/schemaorg-current-https.jsonld — entire vocab as one JSON-LD @graph.
  • Pin to V30.0 (released 2026-03-19) for reproducible codegen: https://schema.org/version/30.0/schemaorg-current-https.jsonld.
  • schema-dts (Google) is not the JSON — it emits TypeScript .d.ts; its schema-dts-gen downloads this same JSON-LD. We transform the JSON-LD directly.
  • Structural fact: the dump is a flat graph — Property nodes point at Class nodes via domainIncludes (+ rangeIncludes for value types); they are NOT nested under classes. Per-type property lists require inverting properties onto their class and resolving subClassOf.

Package design

  • Name: @0xintuition/schema-org (provisional; distinct from the root schema/ hosting dir).
  • Scope: ALL ~820 types — the complete foundation, not just the entities we currently classify.
  • Layout (matches classifications / predicates): one file per type under src/generated/types/<Type> + src/generated/index.ts aggregator + typed accessors (getType, getPropertiesFor). Subpath exports ("./*") so @0xintuition/schema-org/Book resolves.
  • Public, MIT, alpha tag — same build/publish tooling (tsc -p tsconfig.build.json → dist, pack-release.mjs).

Build pipeline

  1. Fetch pinned V30.0 JSON-LD (record the version in the repo).
  2. Parse @graph; split rdfs:Class vs rdf:Property nodes.
  3. Invert: group properties by each class in their domainIncludes; carry rangeIncludes, rdfs:comment, rdfs:label.
  4. Resolve subClassOf chains per type.
  5. Emit one artifact per type + index + accessors.
  6. Vitest: assert inheritance resolution (e.g. Bookname from Thing, author from CreativeWork).

Data shape — decided: defer the flattened view

  • Start normalized: each type carries its own directly-declared properties + its subClassOf chain. This falls out of the scrape for free.
  • Flattened-with-provenance (all props, each tagged with originating ancestor) is a derived second pass generated by walking the chain — added later if/when consumers need it. No decision owed now.

Open / deferred

  • Per-type artifact as raw .json vs typed .ts const wrapper (classifications uses typed .ts consts — likely match for DX).
  • Whether classifications takes a hard dependency on this package, or cross-checks at build time only.
  • Pinned-version fetch + update mechanism.
  • Final naming (schema-org vs schemaorg).

Acceptance criteria

  • @0xintuition/schema-org builds + typechecks + publishes (alpha) via existing tooling.
  • All current schema.org types codegenned, one artifact per type, subpath-exported.
  • Each type exposes own properties + subClassOf chain; accessor resolves inherited properties with provenance.
  • Pinned to a specific schema.org version with a reproducible regen script.
  • Tests assert inheritance resolution (Bookname from Thing, author from CreativeWork).

Dependencies & sequencing

Foundation layer — blocks ENG-12256 (predicates backfill uses the schema.org matrix to find missing predicates), which in turn blocks ENG-12257 (classifications metadataPredicates). Parent: ENG-12195. Gate: second review / align-with-Billy before finalizing the shape (per jam); MK data work paused until the foundation is locked.

References

@jonathanprozzi
jonathanprozzi marked this pull request as ready for review June 2, 2026 19:33
@jonathanprozzi
jonathanprozzi merged commit d06216e into main Jun 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant