Merge pull request #130 from nfdi-de/Improve-docs2

Add proper docs

Author: HendrikBorgelt

docs/about.md

@@ -1,3 +1,53 @@
-# ChemDCAT-AP
+# About
 
-This is an extension of the DCAT Application Profile in LinkML. It is intended to be used by NFDI4Chem & NFDI4Cat as a core that can further be extended in profiles to provide domain specific metadata for a dataset.
+## Motivation
+
+Chemistry data repositories hold thousands of datasets, but finding the right one is hard. A researcher looking for _"all 13C NMR spectra of thiazole derivatives measured above 290 K"_ or _"all Suzuki couplings that used a palladium catalyst and achieved yields above 80%"_ cannot express these queries against plain DCAT-AP metadata. DCAT-AP describes datasets with free-text titles, descriptions, simple keywords tags and coarse theme vocabularies. It currently has no structured way to say what analysis method generated a dataset, which molecule was studied, what reaction type was performed, which instrument was used, or what parameters were applied.
+
+ChemDCAT-AP exists to make such queries possible. By structuring the metadata about *how* a dataset was generated and *what* it is about, it enables domain-specific faceted search within data repositories. Concrete examples of filters this enables:
+
+- **By analysis method** → Find all datasets generated by heteronuclear single quantum coherence (HSQC) NMR spectroscopy, or by gas chromatography-mass spectrometry, or by X-ray powder diffraction.
+- **By analysed substance** → Find all datasets about a specific molecule (via InChIKey), or about any compound containing a thiazole substructure (via SMILES substructure search on the `composed_of` chain).
+- **By reaction characteristics** → Find all datasets about reactions that used a specific catalyst, produced a specific product, or achieved a yield above a threshold.
+- **By instrument parameters** → Find all NMR datasets acquired with a specific pulse sequence, at a specific excitation frequency, or on a specific spectrometer model.
+
+These filters require structured, machine-actionable metadata at a level of detail that DCAT-AP alone cannot provide. DCAT-AP+ adds the generic provenance and attribute machinery that makes this expressivity possible, but using it for chemistry requires manually classifying every instance with the correct ontology term via `rdf_type` (e.g., typing a qualitative attribute as `CHEMINF:000059` to indicate it is an *InChIKey*). ChemDCAT-AP bakes such domain knowledge into the schema: it provides intuitive subclasses and dedicated slots with the right ontology mappings already in place, so that developers and data stewards can produce precise chemistry metadata without needing to look up the correct ontology terms themselves.
+
+## Origin
+
+ChemDCAT-AP grew out of a collaboration between [NFDI4Chem](https://nfdi4chem.de) (NFDI for Chemistry) and [NFDI4Cat](https://nfdi4cat.org/) (NFDI for Catalysis-Related Sciences). Both consortia needed fine-grained, machine-actionable metadata for domain-specific search in their data repositories, and given the overlap between chemistry and catalysis (e.g. both need chemical identifiers, characterization methods, reaction descriptions, ...) a joint effort was the natural choice. From the start it was clear that a DCAT-AP extension for this scope would need to be modular. As the schema took shape, it became apparent that its upper layer — structured provenance, generic attribute patterns, flexible classification — was entirely domain-agnostic and could serve as a reusable module for other domains. This layer was spun off as [DCAT-AP+](https://nfdi-de.github.io/dcat-ap-plus/), and ChemDCAT-AP became the first domain profile built on top of it.
+
+Consequently, ChemDCAT-AP serves a dual role: 
+
+* it will be implemented and further extended for **production metadata schemata** used by NFDI4Cat and NFDI4Chem services, such as [NFDI4Chem's Search Service](https://search.nfdi4chem.de/) or NFDI4Cat's [Repo4Cat](https://repository.nfdi4cat.org) and [Metadata4cat](https://nfdi4cat.org/Services/Meta4Cat.html),
+* and a **reference implementation** that demonstrates the [DCAT-AP+ extension rules](https://nfdi-de.github.io/dcat-ap-plus/latest/how-to-extend/) in practice. Every design decision documented here can serve as a template for other domains building their own DCAT-AP+ extension profiles. 
+
+
+
+## Design principles
+
+The collaborative development process between NFDI4Chem and NFDI4Cat followed four core principles:
+
+1. **Conformance** — Following [DCAT-AP's extension guidelines](https://interoperable-europe.ec.europa.eu/collection/semic-support-centre/solution/dcat-application-profile-implementation-guidelines/release-0), all extensions strictly adhere to the mandatory constraints of the official [DCAT-AP 3.0](https://semiceu.github.io/DCAT-AP/releases/3.0.0/) specification.
+2. **Discoverability-centric** — New properties are chosen to directly improve dataset findability for specific chemistry and catalysis use cases (see [Motivation](#motivation)).
+3. **Semantic grounding** — Every added class and property is mapped to an established ontology (PROV-O, QUDT, BFO, IAO, OBI, CHEBI, CHMO, SIO, etc.) via `class_uri` and `slot_uri`.
+4. **Simplicity for data stewards and developers** — The schema remains usable without requiring deep ontology expertise. Dedicated typed slots, clear naming, and the [quick start](quickstart.md) lower the adoption barrier.
+
+For more on DCAT-AP+'s design rationale and the gap in DCAT-AP, see the [DCAT-AP+ design patterns documentation](https://nfdi-de.github.io/dcat-ap-plus/latest/design-patterns/).
+
+## Publication
+
+The design, implementation, and evaluation of DCAT-AP+ and ChemDCAT-AP were presented at the [19th International Conference on Metadata and Semantics Research (MTSR 2025)](https://www.mtsr-conf.org/home), Thessaloniki, Greece. Preprint: [https://doi.org/10.48550/arXiv.2602.01822](https://doi.org/10.48550/arXiv.2602.01822).
+
+## Funding
+
+This work is funded by the German Research Foundation (DFG) as part of the National Research Data Infrastructure (NFDI):
+
+| Project | DFG Grant | Link |
+|---|---|---|
+| **NFDI4Chem** — NFDI for Chemistry | [441958208](https://gepris.dfg.de/gepris/projekt/441958208) | [nfdi4chem.de](https://nfdi4chem.de) |
+| **NFDI4Cat** — NFDI for Catalysis-Related Sciences | [441926934](https://gepris.dfg.de/gepris/projekt/441926934) | [nfdi4cat.org](https://nfdi4cat.org/) |
+
+## License
+
+ChemDCAT-AP is released under [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/). The repository code is licensed under [MIT](https://github.com/nfdi-de/chem-dcat-ap/blob/main/LICENSE).

docs/chemical-entities.md

@@ -0,0 +1,154 @@
+# Chemical entities
+
+The Chemical Entities AP module ([chemical_entities_ap.yaml](schema/chemical_entities_ap.yaml)) imports the Material Entities AP module and thus adds chemical identity to the material entity layer. It allows describing chemical substances as material things that are mixable, soluble and most importantly composed of certain chemical structures.
+
+## Design Pattern
+
+![chemical_entities_dark.svg](images/chemical_entities_dark.svg#only-dark)
+![chemical_entities_light.svg](images/chemical_entities_light.svg#only-light)
+
+## ChemicalEntity
+
+The `ChemicalEntity` shape is mapped to `CHEBI:23367` (labeled _molecular entity_ in CHEBI; aliased as such in the schema) and extends `Entity` from DCAT-AP+, **not** `MaterialEntity`. It was labeled `ChemicalEntity` because chemists often find CHEBI's label misleading as it suggests only molecules, excluding ions, radicals, complexes, and conformers that are all in scope of `CHEBI:23367`.
+
+!!! note "Why ChemicalEntity extends Entity, not MaterialEntity"
+    Ontologically, a CHEBI chemical entity *is* a BFO material entity, and a reasoner can infer this from the ontology axioms. But ChemDCAT-AP is a shape specification, not an ontology. If `ChemicalEntity` extended `MaterialEntity`, it would inherit the `MaterialisticMixin` slots (`has_temperature`, `has_mass`, `has_volume`, `has_density`, `has_pressure`, `has_physical_state`) via LinkML's `is_a` inheritance. These physical property slots do not make sense at the molecular level (an individual molecule does not have a temperature or a density) and would confuse schema users into thinking they should populate them. Extending `Entity` directly avoids this: molecular-level classes get only the slots they need (structure descriptors), while macroscopic substance classes get physical properties through the `ChemicalSubstanceMixin` applied to `MaterialEntity` subclasses.
+
+### Structure descriptor slots
+
+`ChemicalEntity` introduces six slots for chemical identity, all sub-slots of the DCAT-AP+ `has_qualitative_attribute` or `has_quantitative_attribute`:
+
+| Slot                | Range class        | class_uri        | What it captures           |
+|---------------------|--------------------|------------------|----------------------------|
+| `inchi`             | `InChi`            | `CHEMINF:000113` | InChI structure descriptor |
+| `inchikey`          | `InChIKey`         | `CHEMINF:000059` | InChI key descriptor       |
+| `smiles`            | `SMILES`           | `CHEMINF:000018` | SMILES line notation       |
+| `molecular_formula` | `MolecularFormula` | `CHEMINF:000042` | Hill system formula        |
+| `iupac_name`        | `IUPACName`        | `CHEMINF:000107` | Systematic IUPAC name      |
+| `has_molar_mass`    | `MolarMass`        | `AFR:0002409`    | Molar mass quantity        |
+
+The first five are `QualitativeAttribute` subclasses (string-valued descriptors); `MolarMass` is a `QuantitativeAttribute` subclass (numeric-valued with unit). All use `slot_uri: SIO:000008` (has attribute). All descriptor slots are `recommended: true`, meaning they should be provided when available but are not required for validation.
+
+!!! tip "Use dedicated slots, not generic `has_qualitative_attribute`"
+    ChemDCAT-AP defines `inchi`, `smiles`, etc. as dedicated sub-slots with typed ranges. Use these instead of the generic `has_qualitative_attribute` with a `rdf_type` classification. The dedicated slots produce more explicit instance data, enable tighter validation, and are easier to query.
+
+### Compositional structure 
+
+`ChemicalEntity` overrides `has_part` (inherited from `Entity`) to constrain its range to `ChemicalEntity` and its mapping is changed to `BFO:0000051`. This enables recursive molecular composition: a complex can declare its ligands as parts, each with their own structure descriptors.
+
+## ChemicalSubstanceMixin
+
+An abstract mixin that extends `MaterialisticMixin` with chemistry-specific substance properties. It is applied to `MaterialEntity` subclasses that represent macroscopic chemical substances.
+
+| Slot | Range | Inherited from |
+|---|---|---|
+| `has_concentration` | `Concentration` | ChemicalSubstanceMixin |
+| `has_ph_value` | `PHValue` | ChemicalSubstanceMixin |
+| `has_amount` | `AmountOfSubstance` | ChemicalSubstanceMixin |
+| `composed_of` | `ChemicalEntity[]` | ChemicalSubstanceMixin |
+| `has_temperature` | `Temperature` | MaterialisticMixin |
+| `has_mass` | `Mass` | MaterialisticMixin |
+| `has_volume` | `Volume` | MaterialisticMixin |
+| `has_density` | `Density` | MaterialisticMixin |
+| `has_pressure` | `Pressure` | MaterialisticMixin |
+| `has_physical_state` | `PhysicalStateEnum` | MaterialisticMixin |
+| `alternative_label` | `string` | MaterialisticMixin |
+
+The `composed_of` slot (a sub-slot of `has_part`, mapped to `BFO:0000051`) links a substance to its constituent `ChemicalEntity` instances. This is the bridge between the macroscopic substance and the molecular-level identity.
+
+### Chemistry-specific attribute classes
+
+| Class | class_uri | Parent | Mappings |
+|---|---|---|---|
+| `Concentration` | `CHMO:0002820` | `QuantitativeAttribute` | exact: `EDAM:2140`, `NCIT:C41185`, `VOC4CAT:0007244`, `AFR:0002036` |
+| `AmountOfSubstance` | `qudt:Quantity` | `QuantitativeAttribute` | close: `PATO:0000070` |
+| `PHValue` | `SIO:001089` | `QuantitativeAttribute` | exact: `NCIT:C45997`, `AFR:0001142` |
+
+These follow the DCAT-AP+ [QuantitativeAttribute pattern](https://nfdi-de.github.io/dcat-ap-plus/latest/design-patterns/#quantitativeattribute). Each narrows the semantic intent while preserving the structural `value` + `has_quantity_type` + `unit` shape.
+
+!!! warning "Enum bindings for `has_quantity_type` and `unit` are still experimental"
+    The schema declares bindings that constrain `has_quantity_type` and `unit` values to QUDT QuantityKind and Unit vocabularies respectively. These bindings are not enforced by `linkml-validate`. They will be validated using the [linkml-term-validator](https://linkml.io/linkml-term-validator/), which is meant to supports dynamic enums and binding validation. Yet, this still needs to be implemented in the ChemDCAT-AP pipeline.
+
+## SubstanceSample
+
+Mapped to `SIO:001378` (analyte), `SubstanceSample` extends `MaterialSample` (from `material_entities_ap`) and uses the `ChemicalSubstanceMixin`. It is the central class for any evaluated chemical substance in ChemDCAT-AP. It combines three capabilities:
+
+- **Provenance linking** (via `EvaluatedEntity`): it is meant to be used as the range of `evaluated_entity` and `is_about_entity` in specializations of `DataGeneratingActivity` respectively `Dataset` (see [dataset and activity shapes](dataset-activity-shapes.md)).
+- **Physical properties** (via `MaterialisticMixin`, inherited through `ChemicalSubstanceMixin`): temperature, mass, volume, density, pressure, physical state.
+- **Chemical identity** (via `ChemicalSubstanceMixin`): `composed_of` linking to `ChemicalEntity` instances with structure descriptors, plus concentration, pH, and amount of substance.
+
+Use `SubstanceSample` whenever your data describes a chemical substance that was the subject of a measurement or analysis. This covers NMR samples, catalysis test substances, reaction aliquots, and any other analytically characterized chemical material. For substances that participate in a reaction but are not themselves the evaluated subject, use the reaction participant classes (`StartingMaterial`, `Reagent`, etc.) instead.
+
+### Example: a SubstanceSample with chemical identity
+
+From the test dataset, a Chemotion Repository compound:
+
+```yaml
+id: https://dx.doi.org/10.14272/UGRXAOUDHZOHPF-UHFFFAOYSA-N.2
+title: "CRS-50440"
+rdf_type:
+  id: CHEBI:59999
+  title: "chemical substance"
+other_identifier:
+  - notation: https://www.chemotion-repository.net/pid/50440
+has_temperature:
+  - rdf_type:
+      id: NMR:1400025
+      title: "sample temperature in magnet"
+    has_quantity_type: http://qudt.org/vocab/quantitykind/Temperature
+    unit: https://qudt.org/vocab/unit/K
+    value: 300.0
+composed_of:
+  - id: https://dx.doi.org/10.14272/UGRXAOUDHZOHPF-UHFFFAOYSA-N.2#EvaluatedCompound
+    description: "compound assigned to the sample"
+    other_identifier:
+      - notation: https://pubchem.ncbi.nlm.nih.gov/compound/26248854
+    inchikey:
+      - value: "UGRXAOUDHZOHPF-UHFFFAOYSA-N"
+        title: "assigned InChiKey"
+    inchi:
+      - value: "InChI=1S/C11H12N2S/c1-12-7-10-8-14-11(13-10)9-5-3-2-4-6-9/h2-6,8,12H,7H2,1H3"
+        title: "assigned InChi"
+    smiles:
+      - value: " CNCc1csc(n1)c1ccccc1"
+        title: "assigned SMILES"
+    molecular_formula:
+      - value: "C11H12N2S"
+        title: "assigned molecular formula"
+    iupac_name:
+      - value: "N-methyl-1-(2-phenyl-1,3-thiazol-4-yl)methanamine"
+        description: Chemotion IUPAC name
+      - value: "Methyl[(2-phenyl-1,3-thiazol-4-yl)methyl]amine"
+        description: PubChem IUPAC name
+    has_molar_mass:
+      - has_quantity_type: http://qudt.org/vocab/quantitykind/MolarMass
+        unit: https://qudt.org/vocab/unit/GM-PER-MOL
+        value: 204.072119
+        description: Molar mass as specified in the Chemotion repository.
+      - has_quantity_type: http://qudt.org/vocab/quantitykind/MolarMass
+        unit: https://qudt.org/vocab/unit/GM-PER-MOL
+        value: 204.29
+        description: Molar mass as specified in PubChem.
+```
+
+Key observations:
+
+- The `SubstanceSample` (macroscopic level) links to its `ChemicalEntity` (molecular level) via `composed_of`.
+- The `ChemicalEntity` carries all structure descriptors. Each descriptor slot allows multiple values (e.g. two `iupac_name` entries from different sources, two `has_molar_mass` entries with different precision).
+- The sample's temperature uses `rdf_type` to classify it as specifically the sample temperature in the NMR magnet (`NMR:1400025`), not just any temperature. Similarly, the `SubstanceSample` is additionally typed to be an instance of CHEBI's _chemical substance_ class. This is the [ClassifierMixin](https://nfdi-de.github.io/dcat-ap-plus/latest/design-patterns/#pattern-3-flexible-classification-classifiermixin) at work.
+
+## PolymerSample and PolymerMixin
+
+`PolymerSample` extends `SubstanceSample` and mixes in `PolymerMixin`. It follows the same pattern as `SubstanceSample` itself: a substance class gains domain-specific properties through a mixin.
+
+`PolymerMixin` extends `ChemicalSubstanceMixin` but currently defines no additional slots. It is a placeholder for future polymer-specific properties (degree of polymerization, molecular weight distribution, branching, etc.) that will follow the same mixin-based pattern used throughout ChemDCAT-AP.
+
+!!! warning "Stub: shared `class_uri`"
+    `PolymerSample` currently shares `class_uri: SIO:001378` with its parent `SubstanceSample`. A more specific mapping is planned for a future release. The schema flags this as a TODO.
+
+## Atom
+
+`Atom` (mapped to `CHEBI:33250`) extends `Entity` directly. It is a currently minimal shape whose only addition is that `rdf_type` is `required: true`, forcing every instance to declare which specific atom type it is via the [ClassifierMixin](https://nfdi-de.github.io/dcat-ap-plus/latest/design-patterns/#pattern-3-flexible-classification-classifiermixin). The slot description indicates that the value should come from CHEBI's atom branch (`CHEBI:33250` and its subclasses, e.g. `CHEBI:26708` for a sodium atom), but this constraint is not yet enforced programmatically. Future versions of ChemDCAT-AP will use LinkML [slot bindings](https://linkml.io/linkml-model/latest/docs/bindings/) and the [linkml-term-validator](https://linkml.io/linkml-term-validator/) to validate this.
+
+!!! note "Why Atom extends Entity, not ChemicalEntity"
+    Since `ChemicalEntity` is grounded in CHEBI' _molecular entity_, and CHEBI's atom class (`CHEBI:33250`) is not a subclass of molecular entity in CHEBI's class hierarchy, subsuming `Atom` under `ChemicalEntity` would be ontologically incorrect. Extending `Entity` directly fixes this issue.