Schematic v25.7.1
#1631
Merged
Schematic v25.7.1
#1631
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
add command to generate jsonschemas
Main merge back to develop
…1599) add property descriptions to validation jsonschemas
Delete HTAN data model in documentation
* refactored DataModelJSONSchema into DataModelJSONSchema2 * refactored DataModelJSONSchema into DataModelJSONSchema2 * fix test * added minimum, maximum, and description to schemas * formatting changes * ran vlack * fix types that do not work with python3.9 * fix types that do not work with python3.9 * ran black * added test files * remove test files form git control * improve test * git rm test files * renamed class * improved tests * added two componets to JSON Schema test * added two componets to JSON Schema test * renamed test files * git rm test files * component generator now uses new schema generator class * removed description adding from component generator * fix formatting * added missing end of file lines * move test files * sonarcloud fixes * fix types not supported in python3.9 * fix bad path * remove uneeded test * synapse does not allow the type keyword to be an array * add integration test * fix formatting * add tests to cover missing coverage * added title keyword to all types in an anyOf list * regenerated expected test files * fix end of lines * change integration test to use Synapse fixture * Linglings suggestions * removed unused import * JSON schemas not written with display name * change schemas to not use display name * add integration test for validation schemas in Synapse * added json schema validation tests * added missing docstring * moved instances to files * fix bug in schema generation * fix mypy issues * cleaned up test functionality into fixtures * add test case * various improvements * integration tests now create schemas instead of using expected ones * fixed bug where conditiuonal dependencies where using display names * added additional schema validation tests * fixed bug in json_files_equal * files formatted * improved module level docstring * move function to method of PropertyData * improve PropertyData docstring * improvements to JSONSchema class * fix argument name * improved NodeProcessor docstring * ran pre-commit * fixed line in docstring * remove uneeded function * fix accidental commit * fix incorrect type is test * Add WHEN/GIVEN/THEN pattern to some tests * ran pre-commit * added validation rules to exception message * added validation rules to exception message * cleaned up comments, docstring and vareiable name * imrpive docstring * ran pre-commit * Linglings suggestions * ran pre-commit * ran pylint * added tests for _write_data_model * added unit test for DataModelGraphExplorer.get_adjacent_nodes_by_relationship * revert json_files_equal back to orginal state * created a metadata_model fixture in conftest * ran pre-commit * change name of output directory, possible collision with other tests * refactored JSONSchemaGenerator class into function * JSONSchemaComponentGnerator now using crate_json_schema function * ran pre-commit * can now write using display names * added attributes to example model * moved functioanlity for creating nodes to the Node class * moved functioanlity for creating nodes to the Node class * cleaned up formatting * remove dmr from classes * clean up tests * use new display names parameter * run pre-commit * move module-scoped synapse to conftest * remove unused imports * add more detail to test id * add ids to paramatized tests * fix malformed test ids * ran pre-commit * removed after yield teardowns with finalizers * ran pre-commit * cleanup fixture types * Update tests/integration/test_create_json_schema.py Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com> * randomize test json folder output * Update tests/unit/test_create_json_schema.py Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com> * clarified error message with validation rule * improved logging * file cleaneup * remove fstrings in logging functions * improve docstrings in test files * rename test for clarity * change fixtures to globals * added error handling to update_property method * changed len(all_of) == 0 to not all_of * moved all new components into new datatype * improved typing * fix test so it runs better in parallel * tagged some problematic tests as single process * tagged test as single process * impoved typing * ran pre-commit * fix tests to handle new component in example data model * ran pre-commit * tagged some problematic tests as single proccess * ran pre-commit * tag tests as single process * tag tests as single process * tag tests as single process * tag tests as single process * mark problematic test as local only * mark problematic test as local only * unmark tests as single process * ran pre commit * remove single process test tag * tag tests as local only * tag test as single process --------- Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>
This reverts commit bf2b1ee.
Tests all passed locally.
* added new test model with sms:columType * added dmr fixture * added new relationship, and mehtod for getting allowed values * JSONLD parser now parses sms:columType field * revert typing back to previous typing * add mising return type * made return type more specific * combined two if statements * ran black * move allowed values method to mixin * fix mypy error * address pylint errors * move unpacking functionality to parser class * move unpacking to utils * add test * fix pylint errors * use util * correct check * correct assertion: * fix test to take into accoutn new relationship * move mixin to util fxn * move existing test to different file * pass in dmr * add in type hinting * update docstring * pass in dmr for tests * move exceptions test * add new test data models * add functinality to get unpacked df of model * wip add tests for new functionality * remove staticmethod * update col name * update tests * add column type to attr:rel dict * add assertion * add messages * fix mypy error * ignore mypy error * pylint errors * fix mypy error * add functinality to get unpacked df of model * remove staticmethod * pass in dmr * correct fxn call * update test * change spy import * run precommit * add check for presense of id in each JSONLD entry * ran pre-commit --------- Co-authored-by: GiaJordan <gianna.jordan@sagebase.org>
…s or display names (#1606) Allow generation of jsonschema files with class labels or display names
* DMGE has method for getting column type from node * refactored some functionality into different methods * cleaned up docstrings * have other methods in DMGE use method * cleanup typos * have class use dmr.get_relationship_value instead of accessing dict directly * ran pre-commit * change how strings are tested * fix type in method name * fix typos * ran pre-commit * changed literal into enum * fixes to csv data model parser so data model graph can parse Column… (#1617) * fixes to csv data model parser so data model grapsh cand parse ColumnType * ran pre-commit * add csv datamodel tests to DMGE for column types * ran pre-commit * remove two accidentally commited files * remove two accidentally commited files * remove print statement * lingling suggestions * fix bad test expected error msg
…ation (#1619) add tests for `DataModelJsonLD` parsing `columnType` info
* refactored DataModelJSONSchema into DataModelJSONSchema2 * refactored DataModelJSONSchema into DataModelJSONSchema2 * fix test * added minimum, maximum, and description to schemas * formatting changes * ran vlack * fix types that do not work with python3.9 * fix types that do not work with python3.9 * ran black * added test files * remove test files form git control * improve test * git rm test files * renamed class * improved tests * added two componets to JSON Schema test * added two componets to JSON Schema test * renamed test files * git rm test files * component generator now uses new schema generator class * removed description adding from component generator * fix formatting * added missing end of file lines * move test files * sonarcloud fixes * fix types not supported in python3.9 * fix bad path * remove uneeded test * synapse does not allow the type keyword to be an array * add integration test * fix formatting * add tests to cover missing coverage * added title keyword to all types in an anyOf list * regenerated expected test files * fix end of lines * change integration test to use Synapse fixture * Linglings suggestions * removed unused import * JSON schemas not written with display name * change schemas to not use display name * add integration test for validation schemas in Synapse * added json schema validation tests * added missing docstring * moved instances to files * fix bug in schema generation * fix mypy issues * cleaned up test functionality into fixtures * add test case * various improvements * integration tests now create schemas instead of using expected ones * fixed bug where conditiuonal dependencies where using display names * added additional schema validation tests * fixed bug in json_files_equal * files formatted * improved module level docstring * move function to method of PropertyData * improve PropertyData docstring * improvements to JSONSchema class * fix argument name * improved NodeProcessor docstring * ran pre-commit * fixed line in docstring * remove uneeded function * fix accidental commit * fix incorrect type is test * Add WHEN/GIVEN/THEN pattern to some tests * ran pre-commit * added validation rules to exception message * added validation rules to exception message * cleaned up comments, docstring and vareiable name * imrpive docstring * ran pre-commit * Linglings suggestions * ran pre-commit * ran pylint * added tests for _write_data_model * added unit test for DataModelGraphExplorer.get_adjacent_nodes_by_relationship * revert json_files_equal back to orginal state * created a metadata_model fixture in conftest * ran pre-commit * change name of output directory, possible collision with other tests * refactored JSONSchemaGenerator class into function * JSONSchemaComponentGnerator now using crate_json_schema function * ran pre-commit * can now write using display names * added attributes to example model * moved functioanlity for creating nodes to the Node class * moved functioanlity for creating nodes to the Node class * cleaned up formatting * remove dmr from classes * clean up tests * use new display names parameter * run pre-commit * move module-scoped synapse to conftest * remove unused imports * add more detail to test id * add ids to paramatized tests * fix malformed test ids * ran pre-commit * removed after yield teardowns with finalizers * ran pre-commit * regex validation rule is now translated to pattern keyword * update expected file * cleanup * cleanup tests * cleanup tests * changed strings into enum classes * moved enums and constants to constants file * ran precommit * handle merge * Linglings suggestions * moved code involving validation rules oout of pont_init and into helper function * added test * add pylint diable line * ran black
consolidate jsonschema/columntype data models
add documentation for jsonschema - data model mappings
jsonschema generation documentation updates
…format:uri based on date and url validation rules (#1620) * add datatypes to data model * add date and url as validaiton ruels for JSON Schemas * ran pre-commit * reordered constant file * fix bug with misnamed variable * cleaned up variable names * some cleanup * clear up docstring * remove bool validaiton rule * clear up docstring * clear up docstring * change _VALIDATION_RULES to dict * redo how type is determined * removed is_type_rule attribute * fixed docstring * refactor * updated columntype datamodels * rename module * added links to JSON Schema docs * ran pre-commit * fix missing incompatible rules * refactored how validation rule inputs are checked * refactored how validation rule inputs are checked * ran pre-commit * variosu suggestions * linglings suggestions * fix tests * giannas suggestions * run pre commit * linglings suggestions * Update schematic/schemas/json_schema_validation_rule_functions.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update schematic/schemas/json_schema_validation_rule_functions.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update schematic/schemas/json_schema_validation_rule_functions.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ype attribute (#1625) * add ability for JSON Schema type to bet set by columnType attribute * clean up docstrings * ran pre-commit
… creating a f… (#1616) * added functions for uploading JSON Schems to Synapse and creating a fileview * ran pre-commit * add missing return statement * refactored functions * refactored functions * fix tests after refactor * improved docstrings * add deprecated to toml and lock file * change wiki to be ownded by folder * split wiki function into two functiosn for updating/creating * added warnings * removed file that should not have been committed * fix docstrings * changed warning message * change how entities are named * change name of variable * fix docstrings * Update schematic/schemas/data_model_graph.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/integration/test_json_schema_functions.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * copilot suggestions * removed parent id as parameter * Linglings suggestions * switch finalizer for cleanup fixture * modify test to handle merge from develop --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* convert csv to jsonld * deleted accidentally commited file * ran precomit * fix inavlid cloum type json
* fix typing for data model graph method * fix data model graph method * handle merge conflict * add missing properties to data model for consistency * add dmge using column type data model * deleted file accidentally git commited * added component_jsonschemas to git ignore file * ran pre-commit
* fix bug where there no validation rules set * update_invalid data models * ran pre-commit
andrewelamb
approved these changes
Jul 15, 2025
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Schematic release
v25.7.1