Create LDH container scaffolding via app/ + LDH CLI, not bulk-loaded trigs

The bulk-loaded container documents rendered no children listings: LDH only
renders one when the container document itself carries an rdf:_1 block
pointing at an ldh:Object with rdf:value ldh:ChildrenView. Scaffolding is
now a LinkedDataHub-Apps-style app/ tree — one Turtle file per container
(root + 9 top-level containers + 7 taxonomy scheme containers, schemes
keeping their skos:ConceptScheme as #this) — PUT over HTTP by `make install`
via the LDH CLI (put.sh + Jena turtle from a ../LinkedDataHub checkout,
LDH_HOME-overridable). LDH adds sioc:has_parent, dct:created/modified and
acl:owner server-side and strips any client-sent sioc triples.

ETL datasets now contain only dh:Item documents, still bulk-loaded with
`make load`, attaching to the scaffolding via unchanged sioc:has_container
URIs: the etl/containers/ domain is gone and the 7 taxonomy mappings no
longer emit scheme graphs. validate.sh is strict again (dct:title +
foaf:primaryTopic on every graph); SHACL keeps only the :ItemShape as
LDH-specific, and :ConceptShape's scheme references switch from sh:class to
host-agnostic IRI patterns since the scheme entity left the files.

Also fix a stale-base trap: static-CSV taxonomy outputs kept the previous
BASE because make skips missing intermediate cache/*.nt when the .trig looks
up to date (this had committed localhost-based trigs for the 4 static
schemes). A cache/.base stamp now invalidates the chain when BASE changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: namedgraph

CLAUDE.md

@@ -19,6 +19,8 @@ uv run --project etl/tools ltlod-reconcile <admin-units|persons> --input … --o
 
 make up                        # deploy LinkedDataHub at https://localhost:4443/ (root Makefile;
                                # bootstraps secrets + server cert, then docker compose up -d)
+make install                   # one-time: PUT app/ scaffolding (root + containers + taxonomy
+                               # schemes) via LDH CLI; needs ../LinkedDataHub checkout (LDH_HOME=…)
 make load                      # bulk-load datasets/current/*/*.trig into fuseki-end-user TDB2;
                                # regenerate with `make -C etl` first (committed data has prod base);
                                # ends with `make public` (anonymous read, LDH make-public.sh equivalent)
@@ -41,10 +43,9 @@ Every domain runs the same four stages (shared scripts in `etl/lib/`):
    (`mappings/*.rq`, `$base`-parameterized) executed by `arq` → TriG. The `.rq`
    files are reusable verbatim as LinkedDataHub CSV imports.
 4. **validate** — `riot --validate` + every graph must have `dct:title` and
-   `foaf:primaryTopic` on the graph URI (`foaf:primaryTopic` waived for
-   `dh:Container` docs; see `etl/lib/validate.sh`) + SHACL shapes per entity
-   type (`etl/shapes/<domain>.ttl`, auto-selected by output dir, executed via
-   `etl/lib/shacl.sh`; also run in CI on committed datasets
+   `foaf:primaryTopic` on the graph URI (see `etl/lib/validate.sh`) + SHACL
+   shapes per entity type (`etl/shapes/<domain>.ttl`, auto-selected by output
+   dir, executed via `etl/lib/shacl.sh`; also run in CI on committed datasets
    by `.github/workflows/shacl-validation.yml`).
 
 Post-ETL: `ltlod-reconcile` matches entities to Wikidata (closed candidate sets
@@ -59,11 +60,12 @@ merge on load). Unmatched entities go to `cache/unmatched*.csv`, never force-mat
   keys** from the source registry (AR codes, JAR codes, Seimas `asmens_id`) — any
   pipeline mints cross-links from bare foreign keys. Single source of truth:
   `etl/URI-SCHEME.md` — update it when adding containers.
-- **LDH document hierarchy**: every document is `a dh:Item` +
-  `sioc:has_container <its-container>`; container docs (`etl/containers/` →
-  `datasets/current/containers/containers.trig`) and taxonomy schemes are
-  `dh:Container` + `sioc:has_parent`. This is what makes LinkedDataHub
-  container pages list children — add both triples in any new mapping.
+- **LDH document hierarchy**: ETL outputs are *only* `dh:Item` docs with
+  `sioc:has_container <its-container>` — add both triples in any new mapping.
+  Containers and taxonomy scheme docs come from `app/` (one Turtle file per
+  container, PUT via LDH CLI by `make install`); each carries an
+  `rdf:_1 <#select-children>` → `ldh:Object`/`ldh:ChildrenView` block, without
+  which LDH renders no children listing at all.
 - **Vocabulary cascade**: W3C specs first → domain-specific third-party vocabs
   (EU SEMIC, OP authority tables, FOAF) → schema.org as general fallback → custom
   (`http://linkeddata.lt/ns#`) last. Rationale per domain: `etl/ONTOLOGY-NOTES.md`.
@@ -101,6 +103,11 @@ merge on load). Unmatched entities go to `cache/unmatched*.csv`, never force-mat
 - Seimas API params: `kadencijos_id` (not `p_kade_id`); position rows may reference
   units absent from current feeds (dissolved commissions, the Board) — org-units
   mapping derives those from the members feed.
+- **Switching `BASE` auto-invalidates static-CSV taxonomies** via the
+  `cache/.base` stamp in `etl/taxonomies/Makefile` (normalize outputs embed
+  the base URI; fetched domains are immune — fetch is FORCE'd). Note `make
+  clean` alone does NOT force the rebuild: missing `cache/*.nt` are
+  intermediate files, which make skips when the `.trig` looks up to date.
 - **`make load` bypasses LDH's HTTP API**: it runs `tdb2.tdbloader` directly against
   the end-user TDB2 store via the one-off `tdb-loader` compose service (the
   `atomgraph/fuseki` image bundles the full Jena CLI inside the fuseki-server jar).
@@ -110,6 +117,11 @@ merge on load). Unmatched entities go to `cache/unmatched*.csv`, never force-mat
 - **Fuseki ports are never published to the host** — query via
   `https://localhost:4443/sparql` or from inside the network:
   `docker compose exec linkeddatahub curl http://varnish-end-user/ds/`.
+- **LDH strips client-sent `sioc:has_parent`/`sioc:has_container` on PUT** and
+  manages the hierarchy itself (re-adds `sioc:has_parent` for `dh:Container`,
+  adds `dh:Item` + `sioc:has_container` otherwise, plus `dct:created`/
+  `acl:owner`) — never put sioc triples in `app/*.ttl`. `make install` is
+  idempotent: PUT replaces the whole named graph.
 - **Public read access is class-based**: `make public` grants
   `acl:accessToClass def:Root, dh:Container, dh:Item, nfo:FileDataObject` —
   ETL documents match because mappings type them `dh:Item`/`dh:Container`.

Makefile

@@ -1,6 +1,18 @@
 SHELL := /bin/bash
 
-.PHONY: up down stop logs cert secrets load public drop
+.PHONY: up down stop logs cert secrets install load public drop
+
+# LDH CLI checkout (provides put.sh etc.); Jena provides the `turtle` command
+LDH_HOME ?= ../LinkedDataHub
+include etl/config.mk          # for JENA_HOME (its BASE is unused here)
+include .env                   # PROTOCOL/HOST/HTTPS_PORT/HTTPS_CLIENT_CERT_PORT/ABS_PATH
+
+ifeq ($(HTTPS_PORT),443)
+BASE_URI  := $(PROTOCOL)://$(HOST)$(ABS_PATH)
+else
+BASE_URI  := $(PROTOCOL)://$(HOST):$(HTTPS_PORT)$(ABS_PATH)
+endif
+PROXY_URI := $(PROTOCOL)://$(HOST):$(HTTPS_CLIENT_CERT_PORT)$(ABS_PATH)
 
 SECRET_FILES := secrets/owner_cert_password.txt \
                 secrets/secretary_cert_password.txt \
@@ -32,6 +44,22 @@ stop:
 logs:
 	docker compose logs -f linkeddatahub
 
+# Create/update the container scaffolding (root + containers + taxonomy scheme
+# docs from app/) via LDH CLI PUTs. Run once after `make up`; re-running is
+# safe (PUT replaces). Order: make up -> make install -> make load.
+install:
+	@[ -d "$(LDH_HOME)/bin" ] || \
+		{ echo "ERROR: LDH CLI not found — clone https://github.com/AtomGraph/LinkedDataHub to $(LDH_HOME) or pass LDH_HOME=…"; exit 1; }
+	@[ -n "$$(docker compose ps -q linkeddatahub)" ] || \
+		{ echo "ERROR: linkeddatahub container not found — run 'make up' first."; exit 1; }
+	@echo "Waiting for LinkedDataHub health (first-boot seeding must finish)..."
+	@until [ "$$(docker inspect -f '{{.State.Health.Status}}' $$(docker compose ps -q linkeddatahub))" = "healthy" ]; do \
+		sleep 5; echo "  ...waiting"; \
+	done
+	PATH="$$(find "$$(cd $(LDH_HOME) && pwd)/bin" -type d | tr '\n' ':')$(JENA_HOME)/bin:$$PATH" \
+		./app/install.sh "$(BASE_URI)" ssl/owner/cert.pem \
+		"$$(cat secrets/owner_cert_password.txt)" "$(PROXY_URI)"
+
 # Bulk-load datasets/current/*/*.trig into the end-user TDB2 store. APPEND-ONLY:
 # clean rebuild = `make down && rm -rf fuseki/end-user && make up && make load`.
 load:

README.md

@@ -150,6 +150,7 @@ du Fuseki, Varnish kešai).
 
 ```shell
 make up      # sugeneruoja slaptažodžius + serverio sertifikatą ir paleidžia LDH
+make install # vienkartinis: sukuria konteinerių dokumentus per LDH CLI (reikia ../LinkedDataHub)
 make -C etl  # perkuria rinkinius su numatytąja baze https://localhost:4443/
 make load    # užkrauna datasets/current/*/*.trig tiesiai į triplestore
 ```
@@ -163,11 +164,22 @@ sugeneruoti su produkcine baze `https://linkeddata.lt/`, tad prieš `make load`
 reikia perkurti su numatytąja lokalia baze (`make -C etl`) — kitaip dokumentų URI
 nesutaps su LDH adresu ir jie nebus pasiekiami.
 
-`make load` duomenis rašo **tiesiogiai į `fuseki-end-user` TDB2 saugyklą**
-(`tdb2.tdbloader` per vienkartinį `tdb-loader` konteinerį) — ne po vieną dokumentą per
-HTTP, kaip daro LDH CLI įrankiai: ~1 mln. ketvertų užsikrauna per kelias minutes.
-Pabaigoje suteikiama vieša skaitymo prieiga (`make public` — LDH CLI `make-public.sh`
-atitikmuo, vykdomas tiesiogiai per `fuseki-admin` konteinerių tinkle).
+Duomenų struktūra kuriama dviem lygiais:
+
+- **Karkasas** (`make install`): šakninis dokumentas, konteineriai ir taksonomijų
+  schemos iš `app/` katalogo dokumentas po dokumento **per LDH CLI** (`put.sh`,
+  kaip [LinkedDataHub-Apps](https://github.com/AtomGraph/LinkedDataHub-Apps)
+  projektuose) — taip dokumentai gauna `ldh:ChildrenView` bloką, dėl kurio
+  konteinerių puslapiai rodo vaikų sąrašus. Reikia šalia išklonintos
+  [LinkedDataHub](https://github.com/AtomGraph/LinkedDataHub) repozitorijos
+  (`../LinkedDataHub`, keičiama per `make install LDH_HOME=…`).
+- **Duomenys** (`make load`): ETL rinkiniai — vien `dh:Item` dokumentai su
+  `sioc:has_container` nuorodomis į karkasą — rašomi **tiesiogiai į
+  `fuseki-end-user` TDB2 saugyklą** (`tdb2.tdbloader` per vienkartinį
+  `tdb-loader` konteinerį), ne po vieną dokumentą per HTTP: ~1 mln. ketvertų
+  užsikrauna per kelias minutes. Pabaigoje suteikiama vieša skaitymo prieiga
+  (`make public` — LDH CLI `make-public.sh` atitikmuo, vykdomas tiesiogiai per
+  `fuseki-admin` konteinerių tinkle).
 Triplestore prievadai **neatveriami į host'ą** — SPARQL užklausos teikiamos per LDH:
 <https://localhost:4443/sparql>. Krovimas yra *append-only*: pakartotinis `make load`
 tik papildo saugyklą; švariam perkrovimui:
@@ -188,9 +200,9 @@ Pastabos:
   `datasets/owner`, `datasets/secretary`) — `datasets/current/` niekada neliečiamas.
 - Prievadai 81/4443/5443 sutampa su kitų lokalių LDH diegimų (pvz., `LinkedDataHub`
   repozitorijos) prievadais — vienu metu gali veikti tik vienas stack'as.
-- Dokumentai naršomi per LDH konteinerius: kiekvienas dokumentas yra `dh:Item` su
-  `sioc:has_container`, o patys konteineriai (`datasets/current/containers/`) —
-  `dh:Container` dokumentai (pvz., <https://localhost:4443/admin-units/>).
+- Dokumentai naršomi per LDH konteinerius: kiekvienas ETL dokumentas yra `dh:Item`
+  su `sioc:has_container`, o konteineriai — LDH CLI sukurti `dh:Container`
+  dokumentai iš `app/` (pvz., <https://localhost:4443/admin-units/>).
 
 ## Ką jau galima atsakyti?
 

app/.ldhignore

@@ -0,0 +1 @@
+root.ttl

app/admin-units.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Administraciniai vienetai"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/install.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Creates/updates the LTLOD container scaffolding (root document + containers +
+# taxonomy scheme containers) on a running LinkedDataHub instance via the LDH
+# CLI. Requires LinkedDataHub's bin/ subdirs and Jena's bin/ (for `turtle`) on
+# $PATH — `make install` in the root Makefile sets this up.
+# Deliberately does NOT call make-public.sh: `make load` ends with `make public`.
+set -euo pipefail
+
+if [ "$#" -ne 3 ] && [ "$#" -ne 4 ]; then
+  echo "Usage:   $0" '$base $cert_pem_file $cert_password [$proxy]' >&2
+  echo "Example: $0" 'https://localhost:4443/ ./ssl/owner/cert.pem Password https://localhost:5443/' >&2
+  echo "Note: special characters such as $ need to be escaped in passwords!" >&2
+  exit 1
+fi
+
+base="$1"
+cert_pem_file=$(realpath "$2")
+cert_password="$3"
+proxy="${4:-$base}"
+
+app_dir="$(cd "$(dirname "$0")" && pwd)"
+
+printf "\n### Updating root document: %s\n" "$base"
+turtle --base="$base" < "$app_dir/root.ttl" | put.sh \
+    -f "$cert_pem_file" \
+    -p "$cert_password" \
+    --proxy "$proxy" \
+    -t "application/n-triples" \
+    "$base"
+
+printf "\n### Updating container documents\n"
+"$app_dir/update-folder.sh" "$base" "$cert_pem_file" "$cert_password" "$app_dir" "$app_dir" "$proxy"

app/legal-entities.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Juridiniai asmenys"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/org-units.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Seimo padaliniai"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/organizations.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Organizacijos"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/parliamentary-groups.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Parlamentinės grupės"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .