Add schema-driven config generator and bids_path() function#1215
Draft
astewartau wants to merge 5 commits intobids-standard:mainfrom
Draft
Add schema-driven config generator and bids_path() function#1215astewartau wants to merge 5 commits intobids-standard:mainfrom
astewartau wants to merge 5 commits intobids-standard:mainfrom
Conversation
Add a new config_gen module that generates pybids config dicts directly from the BIDS schema (via bidsschematools), eliminating the need to manually maintain JSON config files that duplicate the schema. Key additions: - generate_config(): produces entity lists and path patterns from the schema - ConfigExtension + generate_extended_config(): extension mechanism for downstream tools to layer custom entities/patterns on top - bids_path(): standalone convenience function to build BIDS-compliant paths from entity dicts without requiring a BIDSLayout Also adds bidsschematools as an optional dependency (pip install pybids[schema]) and includes comprehensive tests, including a consistency check that validates static config entity/pattern name agreement.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1215 +/- ##
==========================================
+ Coverage 89.40% 89.57% +0.16%
==========================================
Files 66 72 +6
Lines 7251 8651 +1400
Branches 1145 1071 -74
==========================================
+ Hits 6483 7749 +1266
- Misses 559 626 +67
- Partials 209 276 +67 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Allow add_config_paths() to accept dict values (not just file paths), and update Config.load() to handle dict values from config_paths lookup. This enables downstream tools to register generated configs without needing a JSON file on disk.
…Extension
- When entity_overrides renames an entity (e.g., description → desc),
template variables in all generated patterns are also rewritten
- New inject_entity_segments parameter inserts entity segments into
generated patterns at specified positions (e.g., injecting [_hash-{hash}]
after [_ses-{session}] in all main patterns)
- Injection skips patterns where the target is absent (sidecar patterns)
and deduplicates if the segment already exists
…neration extra_rules allows compact rule dicts (datatypes, suffixes, extensions, entities) instead of full pattern strings. Entities are inherited from schema derivative rules and only deviations need to be specified. Also adds default value support for entity level specs, entity name resolution (short BIDS names work in extra_rules), and position resolution through the entity name map.
When sidecar_split=False, rule_to_path_pattern() includes all extensions (including .json, .tsv) in the main pattern instead of generating a separate sidecar pattern. This is needed for derivative workflows that output multi-file results (.nii + .json) atomically. Thread the parameter through generate_path_patterns, generate_config, and generate_extended_config.
bendhouseart
left a comment
bendhouseart
left a comment
There was a problem hiding this comment.
I'd like to see the scope of this reduced such that it's only focused on bids path template generation. That is to say, I think it's better to move pybids to the schema than to try and move the schema to conform to pybids configs.
Comment on lines
+25
to
+38
| try: | ||
| from bidsschematools.schema import load_schema | ||
|
|
||
| HAS_SCHEMA = True | ||
| except ImportError: | ||
| HAS_SCHEMA = False | ||
|
|
||
|
|
||
| def _require_schema(): | ||
| if not HAS_SCHEMA: | ||
| raise ImportError( | ||
| "bidsschematools is required for schema-driven config generation. " | ||
| "Install it with: pip install pybids[schema]" | ||
| ) |
There was a problem hiding this comment.
I'm okay with requiring bidsschematools as it's already in pyproject.toml.
Suggested change
| try: | |
| from bidsschematools.schema import load_schema | |
| HAS_SCHEMA = True | |
| except ImportError: | |
| HAS_SCHEMA = False | |
| def _require_schema(): | |
| if not HAS_SCHEMA: | |
| raise ImportError( | |
| "bidsschematools is required for schema-driven config generation. " | |
| "Install it with: pip install pybids[schema]" | |
| ) | |
| from bidsschematools import schema |
Comment on lines
+46
to
+49
| _DIRECTORY_ENTITIES = { | ||
| "subject": "{subject}", | ||
| "session": "{subject}{session}", | ||
| } |
There was a problem hiding this comment.
I think this should be created from schema.rules.directories instead.
Comment on lines
+51
to
+55
| # Format → regex capture group mapping | ||
| _FORMAT_PATTERN = { | ||
| "label": "([a-zA-Z0-9+]+)", | ||
| "index": "(\\d+)", | ||
| } |
There was a problem hiding this comment.
These are defined in the schema as well no need to do this here, see src/schmea/objecs/formats.yaml
| rule: dict, | ||
| schema, | ||
| sidecar_split: bool = True, | ||
| ) -> list[str]: |
There was a problem hiding this comment.
One shouldn't have to pass the rule and the schema as they're one in the same. I would propose an alternative such as:
def rule_to_path_pattern(
rule,
sidecar_split: bool = True,
) -> list[str]:
Or even:
def rule_to_path_pattern(
rule
) -> list[str]:
Or or, let the use determine which rule they want out of the list by generating every single one of them:
def rule_to_path_pattern(
rule=None
) -> list[str]:
| parts = [] | ||
|
|
||
| # Directory: sub-{subject}[/ses-{session}]/ | ||
| parts.append("sub-{subject}[/ses-{session}]/") |
There was a problem hiding this comment.
We should be determining these sorts of things from schema.meta.templates instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
config_genmodule that generates pybids config dicts from the BIDS schema (viabidsschematools), replacing hand-maintained JSON where possibleConfigExtensionmechanism for downstream tools to layer custom entities, path patterns, and entity overrides on top of schema-generated configsentity_overridesrenames entities (e.g.,description→desc) and automatically rewrites template variables in all generated patternsinject_entity_segmentsinserts entity segments (e.g.,[_hash-{hash}]) into generated patterns at specified positionsextra_rulesallows compact rule dicts (datatypes, suffixes, extensions, entities) to generate path patterns declaratively—entities are inherited from schema derivative rules and only deviations need to be specifiedextra_rulesaccept both schema keys (description) and short BIDS names (desc)defaultvalues in rules (e.g.,mode: image)bids_path()convenience functionadd_config_paths()accepts in-memory config dicts (not just file paths)Test plan