Make committed TriG base-relative (base applied at load time) - #9
Merged
Conversation
Committed datasets/current/**/*.trig now use base-relative IRIs with no @base (e.g. <persons/84854/#this>) instead of absolute https://linkeddata.lt/… URIs. The base is supplied at load/parse/validate time, so one committed dataset serves any deployment base (localhost dev, linkeddata.lt prod) with no regeneration. RDF stores only absolute IRIs, so the data loaded into Fuseki is identical; only the git serialization changes. The mappings still mint absolute IRIs in-model — a new etl/lib/relativize.sh rewrites arq/rdflib output to relative on write (riot streaming writer + strip @base), leaving external URIs (Wikidata, Commons, lrs.lt, schema.org, EU tables, mailto/tel) absolute. - write: relativize.sh; graphify.sh pipes arq through it; reconcile/photos become base-aware via rdflib publicID and are relativized on write - read: validate.sh/shacl.sh/run.sh take a base, resolving relative→absolute - load: tdb-loader resolves against BASE_URI (.env) via riotcmd.riot before tdb2.tdbloader; make load passes -e BASE_URI - CI (shacl-validation.yml) + docs (CLAUDE.md, README, URI-SCHEME, config.mk) Verified with Jena 5.6.0: round-trip (riot --base=$B --output=nquads) of every relativized file is byte-identical to the prior absolute triples; validation, cross-graph queries, and load-time rebase to both bases pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kodėl
Užfiksuoti
datasets/current/**/*.trigfailai koduodavo diegimo host'ą į kiekvieną URI(
<https://linkeddata.lt/persons/84854/#this>, ~717k kartų). Dėl to bazė būdavo „įkepta“į duomenis: perjungiant
BASE(localhost dev ↔ linkeddata.lt prod) reikėdavo perkurti visąrinkinį, o failai nebūdavo perkeliami.
Kas pakeista
Užfiksuoti failai dabar naudoja santykines URI be
@base(<persons/84854/#this>); bazėpritaikoma krovimo / skaitymo / validavimo metu. Tas pats rinkinys tinka bet kuriai bazei —
perkurti nereikia. RDF saugo tik absoliučias URI, tad į Fuseki užkraunami duomenys identiški —
keičiasi tik git serializacija. Išorinės URI (Wikidata, Commons, lrs.lt, schema.org, ES lentelės,
mailto/tel) lieka absoliučios.
Mapping'ai vis dar generuoja absoliučias URI atmintyje — naujas
etl/lib/relativize.shperrašoarq/rdflib išvestį į santykinę rašymo metu (riot srautinis rašytojas + nuimtas@base).relativize.sh;graphify.shpraleidžiaarqper jį;reconcile/photostampabazei jautrūs per rdflib
publicIDir relativizuojami išvestyje.validate.sh/shacl.sh/run.shpriima bazę ir išsprendžia santykines→absoliučias.tdb-loaderišsprendžia pagalBASE_URI(.env) suriotcmd.riotprieštdb2.tdbloader;make loadperduoda-e BASE_URI.shacl-validation.yml,CLAUDE.md,README.md,URI-SCHEME.md,config.mk.Verifikacija (Jena 5.6.0)
riot --base=$B --output=nquadsiš santykiniofailo == ankstesni absoliutūs trejetai (rūšiuoti) — įskaitant 673k ketvertų
streets.trig.validate.sh/shacl.shpraeina teisingais grafų skaičiais (148 Seimo nariai, 10 apskričių…);cross-graph užklausos (
committee-chairs, 4 lygiųterritorial-chain) grąžina pilnus rezultatus.riotcmd.riotpatvirtintas fuseki image'e.
tam, ką dabar generuoja pipeline'as, ir vienodi tarp Jena 5.6.0/6.1.0.
Pastaba (jau egzistavusi, nesusijusi klaida)
config.mknumatytojiJENA_HOME= 6.1.0, betarq6.1.0 negali įvykdyti quad CONSTRUCT(
No dataset writer for Turtle_pretty) — veikia tik 5.6.0. Aplinka eksportuoja 5.6.0, tad realiainekliūva, bet
make -C etlsu config numatytąja verte nulūžtų. Verta sutvarkyti atskirai.🤖 Generated with Claude Code