Skip to content

feat(directory-client): fill in FDP-related fields during publishing#5797

Merged
hslh merged 99 commits intomasterfrom
feat/directory-client-fdp-support
Mar 12, 2026
Merged

feat(directory-client): fill in FDP-related fields during publishing#5797
hslh merged 99 commits intomasterfrom
feat/directory-client-fdp-support

Conversation

@hslh
Copy link
Contributor

@hslh hslh commented Jan 15, 2026

What are the main changes you did

  • Add functionality to the Directory client to fill some fields in the updated Directory data model. These fields are for FDP-compatibility and are not filled in in the staging areas but are filled automatically.

How to test

  • run the tests with pytest
  • run the dev.py script: first set up a new target Directory schema using the molgenis.csv and demo data from this branch. Make sure the staging areas on the server you're testing on match the NationalNodes table. Run the dev.py. See that data is successfully published. Check RDF output via [server]/[schema]/api/rdf and validate using [server]/[schema]/api/rdf?validate=ejp-rd-vp and [server]/[schema]/api/rdf?validate=fdp-v1.2 . The FDP validation should show a single error.

@hslh hslh marked this pull request as ready for review January 29, 2026 12:35
@hslh hslh requested a review from dtroelofsprins January 29, 2026 12:38
@sonarqubecloud
Copy link

@hslh hslh requested a review from YpeZ February 5, 2026 08:58
@hslh hslh self-assigned this Mar 5, 2026
@hslh hslh removed the request for review from dtroelofsprins March 11, 2026 12:29
Copy link
Contributor

@YpeZ YpeZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests work. Examples work as well

@sonarqubecloud
Copy link

@hslh hslh merged commit 90af944 into master Mar 12, 2026
7 of 8 checks passed
@hslh hslh deleted the feat/directory-client-fdp-support branch March 12, 2026 10:33
connoratrug pushed a commit that referenced this pull request Mar 12, 2026
…5797)

* Add endpoint table

* Add Agent table

* Add Catalog table

* Cleaned up and ordered Catalog.csv to align with FDP specs

* Link Catalog and Biobanks

* Clean up Agent table and align with BBMRI EMX1 FDP

* Use semantic prefixes

* Clean up Endpoint table and align with FDP specs

* Add option to specify Agent type

* Fix capitalization

* Add ldp container structure and attributes

* Change columns to align with molgenis.csv

* Catalog, not NationalNodes

* Don't add fdp-o metadata identifier manually

* Don't 'manually' add endpoint url

* date, not datetime

* No fdp metadata identifier for Endpoint either

* Integrate Agent/Catalog/Endpoint into molgenis.csv

* Add Biobank semantics

* Add semantics for Persons

* Add Collections semantics

* Restructure LDP containers

* dct -> dcterms

* Remove semantics following feedback Vittorio

* Remove vcard tags from Agent

* Agent -> Publishers

* Add Organisations table for vcard contact points

* Point contactPoint to Organisations, not Publishers

* Pluralise FDP tables

* Order molgenis.csv by table name

* Add tag and property to Persons

* Add semantics to Catalogs

* CSV linting

* Add properties to Catalogs table

* Remove default policy

* Add DataServices

* Add table definition

* Add Address table for Organisations.address to conform to vcard standard

* Add dcterms:spatial to Collections.country

* Add issued and modified to Collections

* fix comma

* More commas

* Catalog should be Catalogs

* Capitalisation

* attribute name formatting

* Same

* Add fdp-metadata_identifier attributes to allow for output of fdp-o:metadataIdentifier in semantic mode

* Neater full name computed expression which can handle null values

* Add refback to Endpoints.metadata_catalog as Catalogs.endpoint

* Get endpointURL into semantic mode

* Use only fdp-o:hasSoftwareVersion for Endpoints.version

* Add four attributes to Collections

* Change Collections.publisher to ref

* Add dcterms:hasVersiopn to Endpoints.version for FDP SHACL compliance

* Improve dummy URL

* Re-order columns

* Fix label

* Re-order columns

* Add missing column

* Add demo data for newly added tables and columns

* Bump

* Add required field

* Add RDF config to demo data default settings

* fix LDP membership relation of Catlaogs

* Fix test

* Correct description of dct:issued and dct:modified

* Change themes, access rights and languages from hyperlinks to ontologies

* Ontology formatting got messed up

* Update staging area demo data to match directory schema demo data

* Initial catalog membership

* Initial catalog name retrieval

* Rename catalog_name, add checks on Catalogs table

* Add publisher to collections

* Fill language, license, keywords

* Make Themes conformant to EHDS

* Import from src rather than installed module

* Fix mocker patch paths

* Add test for _set_catalog_membership

* Add test for _set_fdp_collection_fields

* Formatting

* Check Publishers table before setting Collections.publisher

* Make backwards compatible with data model without FDP fields

* changed environment variable names in order to avoid clash with env variable 'username' on UNIX systems

* fixed tests

* Allow Python versions > 3.10 in Pipfile, update Pipfile.lock

---------

Co-authored-by: Ype Zijlstra <info@ypezijlstra.frl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants