Improve the documentation

docs/about.md

@@ -1,28 +1,26 @@
 # About DCAT-AP+
 
-Funded by the German Research Foundantion (DFG) as part of the German National Research Data Infrastucture (NFDI) 
-initiative under the grant numbers [441926934](https://gepris.dfg.de/gepris/projekt/441926934) and [441958208](https://gepris.dfg.de/gepris/projekt/441958208), 
-DCAT-AP+ is being developed in close collaboration between the German research 
-infrastructure projects [NFDI4Chem](https://nfdi4chem.de) and [NFDI4Cat](https://nfdi4cat.org/). 
-
-To allow more fine-grained and semantic searches within their data repositories, 
-both projects had to address the need to also provide detailed chemistry-specific metadata for the research data output
-of their communities. Due to the disciplinary overlap of both projects, their previous collaboration was thus intensified 
-to produce a common metadata schema called [ChemDCAT-AP](nfdi-de.github.io/chem-dcat-ap/), an extension of the 
-[DCAT Application Profile](https://semiceu.github.io/DCAT-AP/releases/3.0.0/) that is based mostly on 
-the [Starting Point Terms](https://www.w3.org/TR/prov-o/#description-starting-point-terms) of the W3C standard the Provenance Ontology (PROV-O). With ChemDCAT-AP, the chemical 
-substances, entities and reactions covered by a dataset as well as the processes, 
-tools and devices that were involved its creation can be described in a semantically uniform way that allows further 
-use-case specific extension.
-This work will be published and presented at the 19th International Conference on Metadata and Semantics Research 
-([MTSR](https://www.mtsr-conf.org/home)) Thessaloniki, Greece, 15 - 19 December 2025. 
-
-Since the underlying basic design patterns of ChemDCAT-AP are domain-agnostic and thus applicable to a much wider 
-range of use cases, the core layer of ChemDCAT-AP, called DCAT-AP+, was decided to be outsourced into its own repository.
-
-## Next Steps
-Within the [NFDI Section Metadata Working Group Ontology Harmonization and Mapping](https://www.nfdi.de/section-metadata/),
-we started to test and discuss the applicability of DCAT-AP+ for the whole NFDI community. Additionally, we currently 
-investigate the feasibility of providing more semantic depth and interoperability by mapping the DCAT-AP+ schema 
-elements to the [NFDIcore ontology](https://nfdi.fiz-karlsruhe.de/ontology/) instead of the PROV-O and DCTerms 
-whenever appropriate.
+## Origin
+
+DCAT-AP+ grew out of the collaborative development of [ChemDCAT-AP](https://nfdi-de.github.io/chem-dcat-ap/), a metadata schema for chemistry and catalysis research data. When [NFDI4Chem](https://nfdi4chem.de) and [NFDI4Cat](https://nfdi4cat.org/) needed to describe *how* their datasets were generated and *what* they were about to allow more fine-grained and semantic searches within their data repositories, they found that [DCAT-AP 3.0](https://semiceu.github.io/DCAT-AP/releases/3.0.0/) provided the right foundation for dataset cataloging but lacked the provenance expressivity required for scientific metadata.
+
+The solution they came up with, a provenance layer built on [PROV-O Starting Point Terms](https://www.w3.org/TR/prov-o/#description-starting-point-terms) with a generic attribute pattern and flexible classification mechanism, turned out to be domain-agnostic. It was therefore extracted into its own repository as **DCAT-AP+**, a reusable building block that any domain can import and specialize.
+
+## Publication
+
+The design, implementation, and evaluation of DCAT-AP+ and ChemDCAT-AP were presented at the [19th International Conference on Metadata and Semantics Research (MTSR 2025)](https://www.mtsr-conf.org/home), Thessaloniki, Greece, 15–19 December 2025. Until the proceedings are published you can cite the preprint: https://doi.org/10.48550/arXiv.2602.01822.
+
+
+## Funding
+
+This work is funded by the German Research Foundation (DFG) as part of the National Research Data Infrastructure (NFDI):
+
+| Project | DFG Grant | Link |
+|---|---|---|
+| **NFDI4Cat** — NFDI for Catalysis-Related Sciences | [441926934](https://gepris.dfg.de/gepris/projekt/441926934) | [nfdi4cat.org](https://nfdi4cat.org/) |
+| **NFDI4Chem** — NFDI for Chemistry | [441958208](https://gepris.dfg.de/gepris/projekt/441958208) | [nfdi4chem.de](https://nfdi4chem.de) |
+
+
+## License
+
+DCAT-AP+ is released under [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/). The repository code is licensed under [MIT](https://github.com/nfdi-de/dcat-ap-plus/blob/main/LICENSE).

docs/automatic-generation.md

@@ -1,35 +1,83 @@
-# Automatic generation of DCAT-AP+
-In order to build an extension of DCAT-AP, a faithful translation of DCAT-AP into a LinkML schema was provided first.
-For this, the JSON-LD serialization of the official DCAT-AP 3.0.0 SHACL shapes ([dcat_ap_shacl.jsonld](src/dcat_ap_shacl.jsonld)) were used, 
-which we downloaded from the DCAT-AP GitHub repository [3.0.0 release folder within the master branch](https://github.com/SEMICeu/DCAT-AP/tree/master/releases/3.0.0/shacl). 
-These SHACL shapes were then processed by the 
-[dcat_ap_shacl_2_linkml.py](https://github.com/nfdi-de/dcat-ap-plus/blob/main/src/dcat_ap_plus/dcat_ap_shacl_2_linkml.py) 
-script to generate two LinkML schemas from it:
-
-  * [dcat_ap_linkml.yaml](src/dcat_ap_plus/schema/dcat_ap_linkml.yaml) - an almost 1:1 translation of the DCAT-AP SHACL shapes to LinkML.
-  * [dcat_ap_plus.yaml](src/dcat_ap_plus/schema/dcat_ap_plus.yaml) - the LinkML representation of DCAT-AP to which we added the additional constraints, 
-  * classes and properties we need for our provenance based extension.
-
-## Automatic Translation of DCAT-AP into LinkML
-The `dcat_ap_shacl_2_linkml.py` module performs the translation by reading the DCAT-AP JSON-LD file and generating 
-corresponding LinkML constructs. DCAT-AP's SHACL node shapes are mapped to either LinkML classes or datatypes depending 
-on whether they target ontology classes or XSD types, with property shapes serving as the basis for class slots. To 
-ensure the new model remains semantically identical to the source, the original term IRIs are retained verbatim in the 
-`class_uri` and `slot_uri` fields, drawn from the SHACL `targetClass` and `path` attributes.
-
-Regarding range definitions, we addressed union ranges differently based on type. Object class unions, such as 
-`dcat:primaryTopic`, were managed using the LinkML 
-[any_of](https://linkml.io/linkml/schemas/advanced.html#unions-as-ranges) feature. Conversely, due to the lack of 
-stable support for datatype unions in LinkML ([see issue](https://github.com/linkml/linkml/issues/1813)), we enforced a 
-stricter interpretation for date-related slots, restricting them to the XSD date datatype. This approach allows us to 
-meet current project needs while expecting full datatype union support in upcoming updates.
-
-To adhere to LinkML's naming convention, we also changed the names of the derived and added slots from camel case to 
-snake case.
-
-## Automatic Extension of DCAT-AP in LinkML
-To produce the DCAT-AP+ extension, we used the same Python script to add the additional constraints, classes and 
-properties we needed, which are described in more detail in the 
-[Design Patterns and Decisions section](design-patterns.md). 
-Consequently, any future changes to DCAT-AP+ must be made in the `dcat_ap_shacl_2_linkml.py` 
-([L412-L991](https://github.com/nfdi-de/dcat-ap-plus/blob/main/src/dcat_ap_plus/dcat_ap_shacl_2_linkml.py#L412-L991)).
+# Automatic Generation of DCAT-AP+
+
+In DCAT-AP+ we do not manually recreate DCAT-AP in LinkML but **auto-generate** it as the base layer from the authoritative SHACL shapes published by SEMIC. This ensures that the LinkML schema stays faithful to the official specification and can be updated systematically when DCAT-AP evolves.
+
+## Why auto-generate?
+
+Manual porting of a complex specification invites drift. The DCAT-AP SHACL shapes define ~25 node shapes with ~150 property shapes, each with cardinality constraints, range definitions, and IRI mappings. Reproducing this by hand would be error-prone and hard to maintain across DCAT-AP releases.
+
+By scripting the translation, we get two guarantees:
+
+1. **Semantic identity**: Every `class_uri` and `slot_uri` in the generated LinkML schema is copied verbatim from the SHACL `sh:targetClass` and `sh:path` attributes. The resulting model is structurally equivalent to the official shapes.
+2. **Reproducibility**: When SEMIC publishes a new DCAT-AP release, re-running the script against the updated SHACL shapes produces an updated base layer, making the delta to our extension layer explicit.
+
+## The pipeline
+
+![conversion_pipeline_dark.svg](images/conversion_pipeline_dark.svg#only-dark)
+![conversion_pipeline_light.svg](images/conversion_pipeline_light.svg#only-light)
+The script produces **two** LinkML schemas from the same input:
+
+| Output                                                  | Purpose                                                                                                                                                                          |
+|---------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [`dcat_ap_linkml.yaml`](schema/dcat_ap_linkml.yaml)     | A near-1:1 translation of the DCAT-AP SHACL shapes into LinkML. Useful as a standalone reusable artifact for anyone wanting DCAT-AP in LinkML without extensions.                |
+| [`dcat_ap_plus.yaml`](schema/dcat_ap_plus.yaml)         | The same base layer **plus** the DCAT-AP+ extension: the provenance core, attribute patterns, and classification pattern described in [Design Patterns](design-patterns.md).     |
+
+## Input: Which SHACL shapes?
+
+The script uses the JSON-LD serialization of the DCAT-AP 3.0.0 SHACL shapes, downloaded from the [SEMIC DCAT-AP repository (`master` branch, `releases/3.0.0/shacl/`)](https://github.com/SEMICeu/DCAT-AP/tree/master/releases/3.0.0/shacl).
+
+!!! warning "SEMIC publishes multiple shape files that differ"
+    The shapes in the `master` branch's `releases/3.0.0` folder differ from those in the [tagged `3.0.0` release](https://github.com/SEMICeu/DCAT-AP/releases/tag/3.0.0) and the `3.0.0` branch. We use the `master` branch version because it is the one linked from the [official specification website](https://semiceu.github.io/DCAT-AP/releases/3.0.0/) and reflects the most recent editorial corrections. See also [DCAT-AP issue #428](https://github.com/SEMICeu/DCAT-AP/issues/428).
+
+## How the translation works
+
+The [`dcat_ap_shacl_2_linkml.py`](https://github.com/nfdi-de/dcat-ap-plus/blob/main/src/dcat_ap_plus/dcat_ap_shacl_2_linkml.py) script iterates over each SHACL node shape in the [JSON-LD file](https://github.com/nfdi-de/dcat-ap-plus/blob/main/src/dcat_ap_plus/dcat_ap_shacl.jsonld) and maps it to a LinkML construct:
+
+**Node shapes → classes or datatypes.** A node shape whose `sh:targetClass` points to an ontology class (e.g. `dcat:Dataset`) becomes a LinkML class. A node shape targeting an XSD datatype (e.g. `xsd:duration`) becomes a LinkML datatype.
+
+**Property shapes → slots.** Each `sh:property` within a node shape becomes a slot on the derived class. Cardinality (`sh:minCount`, `sh:maxCount`), range (`sh:class`, `sh:datatype`), and the property IRI (`sh:path`) are all preserved.
+
+**Naming convention.** Slot names are converted from the DCAT-AP camelCase convention to LinkML's snake_case (e.g. `accessURL` → `access_URL`, `contactPoint` → `contact_point`).
+
+### Handling of union ranges
+
+The DCAT-AP shapes contain two kinds of unions:
+
+- **Object class unions** (e.g. `dcat:primaryTopic` can range over `Dataset`, `DatasetSeries`, `Catalogue`, or `DataService`): handled via LinkML's [`any_of`](https://linkml.io/linkml/schemas/advanced.html#unions-as-ranges) keyword.
+- **Datatype unions** (e.g. the `TemporalLiteral` shape unions `xsd:date`, `xsd:dateTime`, `xsd:gYear`, and `xsd:gYearMonth`): due to a [known LinkML limitation](https://github.com/linkml/linkml/issues/1813), these are conservatively restricted to `xsd:date`. This is a stricter interpretation than the official DCAT-AP shapes and will be relaxed once LinkML supports datatype unions.
+
+!!! note "Shapes that are skipped"
+    The script explicitly ignores `rdfs:Literal` (replaced by LinkML's default `string` range), the `CataloguedResource` union shape (replaced by the `Any` class with `any_of` constraints), and a duplicate `mediaType` shape that appears to be an editorial error in the source.
+
+## What is auto-generated vs. manually authored
+
+| Layer | How it's created | Where in the script |
+|---|---|---|
+| **DCAT-AP base** (classes, slots, datatypes, enums from the official shapes) | Auto-generated by `parse_dcat_ap_shacl_shapes()` | [Lines 1–411](https://github.com/nfdi-de/dcat-ap-plus/blob/main/src/dcat_ap_plus/dcat_ap_shacl_2_linkml.py#L1-L411) |
+| **DCAT-AP+ extension** (provenance core, attributes, `ClassifierMixin`, contextual metadata) | Programmatically added by `build_dcatapplus_linkml()` | [Lines 412–991](https://github.com/nfdi-de/dcat-ap-plus/blob/main/src/dcat_ap_plus/dcat_ap_shacl_2_linkml.py#L412-L991) |
+
+The extension layer is authored *in Python code*, not in raw YAML, so that it builds on top of the same `SchemaBuilder` object that holds the auto-generated DCAT-AP base. This ensures that references between base and extension elements (e.g. making `was_generated_by` mandatory on `Dataset`, or adding slots to `Activity`) are validated at build time.
+
+Elements belonging to the DCAT-AP+ extension are tagged with `in_subset: [domain_agnostic_core]` in the schema, making it easy to distinguish them from the auto-generated DCAT-AP base.
+
+## Re-running the generation
+
+To regenerate both schemas after updating the input SHACL shapes or modifying the extension code:
+
+```bash
+# 1. If DCAT-AP has released new shapes, replace the input file:
+#    Download the updated dcat_ap_shacl.jsonld into src/dcat_ap_plus/
+
+# 2. Run the build script:
+uv run python src/dcat_ap_plus/dcat_ap_shacl_2_linkml.py
+
+# 3. Validate the generated schema and test data:
+uv run linkml-validate tests/data/valid/AnalysisDataset-001.yaml \
+  -s src/dcat_ap_plus/schema/dcat_ap_plus.yaml -C AnalysisDataset
+
+# 4. Regenerate the documentation:
+rm -rf docs/elements/*.md && \
+  uv run gen-doc -d docs/elements src/dcat_ap_plus/schema/dcat_ap_plus.yaml
+```
+
+The CI pipeline (GitHub Actions) runs schema validation and data validation on every pull request, catching regressions automatically.

docs/dcat-ap-plus-users.md

@@ -1,5 +1,41 @@
-# Projects using DCAT-AP+
-The following projects use DCAT-AP+ as a basis for further domain-specific extensions of DCAT-AP
+# Projects Using DCAT-AP+
 
-  - [ChemDCAT-AP](https://nfdi-de.github.io/chem-dcat-ap): A chemistry specific DCAT-AP extension from which DCAT-AP+ was spun off.
-  - [NMR-DCAT-AP](https://nfdi4chem.github.io/nmr-dcat-ap/): An extension of ChemDCAT-AP that formalizes the NFDI4Chem NMR Minimal Information Standard.
+## Active domain profiles
+
+### ChemDCAT-AP
+
+[:octicons-repo-16: nfdi-de/chem-dcat-ap](https://github.com/nfdi-de/chem-dcat-ap) · [:octicons-book-16: Documentation](https://nfdi-de.github.io/chem-dcat-ap)
+
+ChemDCAT-AP is the profile from which DCAT-AP+ was originally extracted. Developed jointly by [NFDI4Chem](https://nfdi4chem.de) and [NFDI4Cat](https://nfdi4cat.org/), it specializes the generic DCAT-AP+ classes for chemistry and catalysis research data. Key specializations include:
+
+- `SubstanceSample` a subclass of `EvaluatedEntity` mapped to SIO
+- Domain-specific sub-slots like `used_catalyst` (`is_a: carried_out_by`, mapped to `RXNO:0000425`) and `generated_product` (`is_a: had_output_entity`, mapped to `RO:0004008`)
+- Dedicated properties for chemical identifiers (`inchikey`, `smiles`) that replace the generic `has_qualitative_attribute` pattern
+
+ChemDCAT-AP is deployed in production within the [NFDI4Chem Search Service](https://search.nfdi4chem.de/), where it powers semantically rich metadata export and the NFDI Chemistry Knowledge Graph.
+
+### NMR-DCAT-AP
+
+[:octicons-repo-16: NFDI4Chem/nmr-dcat-ap](https://github.com/NFDI4Chem/nmr-dcat-ap) · [:octicons-book-16: Documentation](https://nfdi4chem.github.io/nmr-dcat-ap/)
+
+NMR-DCAT-AP extends ChemDCAT-AP to formalize the [MARGARITAS](https://doi.org/10.25504/FAIRsharing.c29400) minimum information standard for NMR spectroscopy. It demonstrates how the DCAT-AP+ → ChemDCAT-AP inheritance chain can be extended further to encode specific MIChI (Minimum Information for a Chemical Investigation) standards. The workflow serves as a validated template for encoding other MIChI standards.
+
+## Cross-domain validation
+
+### NOMAD (materials science)
+
+We are collecting dataset examples from [NOMAD](https://nomad-lab.eu/), a materials science data repository, to test DCAT-AP+ against metadata from a domain adjacent to but distinct from chemistry. These examples are being used to validate that the provenance core pattern generalizes beyond its original chemistry and catalysis context. Further examples from other domains are expected as we present this work more broadly within the NFDI Section (Meta)data, Terminology, Provenance.
+
+## Emerging interest
+
+### NFDIcore alignment
+
+We are collaborating with the developers of the [NFDIcore ontology](https://nfdi.fiz-karlsruhe.de/ontology/) to explore mapping DCAT-AP+ schema elements to NFDIcore classes and predicates in addition to PROV-O, DCTerms and QUDT. NFDIcore is a BFO-aligned ontology that provides narrower, more precisely scoped terms that match the intent of several DCAT-AP+ node shapes. Because DCAT-AP+ uses LinkML's `class_uri` and `slot_uri` for ontology alignment (see [Design Patterns: foundational principle](design-patterns.md#foundational-principle-linkml-classes-as-node-shapes)), swapping or supplementing the underlying ontology mappings does not require restructuring the schema.
+
+### Life sciences (RO-Crate / Schema.org)
+
+In conversations with NFDI colleagues from the life sciences, who use [RO-Crate](https://www.researchobject.org/ro-crate/) and Schema.org for attaching domain-specific metadata to datasets, there is good alignment potential. Their approach shares the activity/process focus that DCAT-AP+ introduces. A formal comparison and possible bridging is a future direction.
+
+## Using DCAT-AP+ in your project?
+
+If you are building a domain profile on DCAT-AP+, see the [extension rules](how-to-extend.md) for the contract between DCAT-AP+ and downstream schemas. We welcome additions to this page — open a PR or [start a discussion](https://github.com/nfdi-de/dcat-ap-plus/discussions).