Phase 3: lokf new — scaffold a publishable Astro knowledge base - #14
Merged
Conversation
The instance-level tabular projection (phase 1 added the schema-level bindings +
relational DDL). A bundle becomes one DataFrame per concept type (the nodes)
plus a single `relations` edge table (source, predicate, target).
- lokf.tables: to_frames (pandas | polars), writers (CSV, Parquet, SQLite), and
external_table_ddl for a BigQuery/Athena lakehouse over the Parquet.
- `lokf tables <bundle> --format {csv|parquet|sqlite|bigquery|athena}` CLI.
- pandas/pyarrow live in a new `tables` extra; the core stays light.
- 9 tests; a toolkit/tables docs page; README + .gitignore updated.
126 tests green.
`lokf new <name>` writes a self-contained repo: a starter LOKF bundle, a MkDocs site + GitHub Pages workflow that publishes it, a justfile driving the toolkit via uvx (no install), and the bundled agent skills. In the spirit of linkml-cookiecutter — idea -> bundle -> published site. - lokf.scaffold + `lokf new` CLI. - New `scaffold-knowledge-base` agent skill (idea -> lokf new -> author -> validate -> publish). - Tests: the scaffolded repo has the expected files, ships the skills, and its bundle projects to valid RDF. test_skills updated for the new skill. - Docs: toolkit/scaffold page; README "Start your own". 130 tests green.
Replace the MkDocs scaffold with a packaged Astro template (src/lokf/templates/kb/) that `lokf new` copies + tokenizes, so a fresh KB gets the full LOKF experience: concept pages with typed-relation panels, an interactive /graph cytoscape browser, and graph.json + graph.jsonld endpoints — base-path-aware so it also works as a GitHub project page under /<repo>/. - scaffold.py: copy the template tree, substitute __KB_*__ tokens (site/base split from base_iri), rename _github/_gitignore dotfiles, harden the target/template guards. - Packaging: exclude web/ (+ node_modules/.astro/dist) from the sdist — it was 132 MB (node_modules leaked) and over PyPI's 100 MB limit; now 383 KB. Wheel still ships the 20 template files. - Tests updated for the Astro tree, token substitution, and base derivation; 133 pass. Verified end-to-end: scaffold -> npm build -> preview (home, concept pages with forward+back relations, /graph, graph.json/jsonld). - Docs / skill / README updated (Astro, not MkDocs).
nicholsn
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3 of 3. Stacked on #13 (base = `feat/lokf-tables`) — GitHub retargets to `main` as the stack merges.
Closes the loop: from an idea to a published knowledge graph. `lokf new ` scaffolds a complete, self-contained repo — in the spirit of linkml-cookiecutter — and the new KB gets the full LOKF experience, not a plain doc site.
`lokf new`
lokf new my-kb --title "My Knowledge Base" --base-iri https://myorg.github.io/my-kb/Writes a repo with:
?focus=deep-links, full-screen), and `graph.json` + `graph.jsonld` endpoints.site/baseare derived from--base-iri, so it works on a domain or as a GitHub project page under/<repo>/.just setup | dev | sitefor the app;serve | rdf | tablesfor the toolkit viauvx), `.claude/skills/` (the bundled agent skills), README, .gitignore.The template lives at
src/lokf/templates/kb/and is copied + token-substituted byscaffold.py; the scaffolded bundle is valid LOKF out of the box.scaffold-knowledge-baseagent skillA new bundled skill that drives idea →
lokf new→ author concepts → validate → publish, handing off toauthor-concept/enrich-relations.Packaging fix (found by adversarial review)
The sdist was 132 MB — hatchling was leaking
web/node_modules(20k files), over PyPI's limit. Excludedweb/and build caches from the sdist → 383 KB; the wheel still ships all 20 template files.Verified
astro buildon a freshly scaffolded KB (4 pages), preview-checked (home, concept pages with forward+back relations, the graph browser renderingWeekly Active Users → measures → Active User,graph.json/graph.jsonld), wheel/sdist inspected, 133 tests green.Completes the arc: #12 bindings + relational schema, #13
lokf.tables, #14 the Astro scaffolder. Merge order: #12 → #13 → #14.