Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update schema references to point to nmdc_materialized_patterns variant #887

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Update schema references to point to nmdc_materialized_patterns variant #887

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e096fc0
Replace misleading schema filename reference in default param value
eecavanna Jan 30, 2025
2436761
Replace obsolete schema filename in Python notebook Markdown prose
eecavanna Jan 30, 2025
1e0e1b2
Replace obsolete schema filename and URL in JSON Validation tutorial
eecavanna Jan 30, 2025
2042c09
Replace hard-coded obsolete file paths with references to CLI args
eecavanna Feb 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/tutorials/json.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ endpoint allows you to get a filtered list of documents from one of the NMDC Sch
The `collection_name` must be one defined for a
[nmdc:Database](https://microbiomedata.github.io/nmdc-schema/Database/), in the form expected by the
JSON Schema,
[nmdc.schema.json](https://github.com/microbiomedata/nmdc-schema/blob/69fd1ee91afac1a943b2cc9bfbfdecd0e2cdd089/jsonschema/nmdc.schema.json#L987).
[nmdc_materialized_patterns.schema.json](https://github.com/microbiomedata/nmdc-schema/blob/1b42cef7e3a47930d25bde35b4bca0aa4391b283/nmdc_schema/nmdc_materialized_patterns.schema.json#L6699).
This typically means that any spaces in the name should be entered as underscores (`_`) instead.

The `filter`, if provided, is a JSON document in the form of the
Expand Down
4 changes: 1 addition & 3 deletions metadata-translation/notebooks/ghissue_252_253_linked_samples.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,9 +301,7 @@
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Need to update mongo collection schema to account for updated fields in nmdc.schema.json:"
]
"source": "Need to update mongo collection schema to account for updated fields in `nmdc_materialized_patterns.schema.json`:"
},
{
"cell_type": "code",
Expand Down
10 changes: 9 additions & 1 deletion metadata-translation/src/bin/validate_json.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@


def validate_json(data_path, schema_path, log_file):
r"""
TODO: Document this function.
TODO: Add type hints for this function's parameters and its return value.
"""

with open(data_path, "r") as json_file: # load data
data = json.load(json_file)
Expand All @@ -32,9 +36,13 @@ def validate_json(data_path, schema_path, log_file):

def test_gold_study_json(
data_path="output/nmdc_etl/gold_study.json",
schema_path="../../../schema/nmdc.schema.json",
schema_path="/path/to/nmdc_materialized_patterns.schema.json",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we not doing this using the python package or linkml. @sierra-moxon please weigh in here but we can't just have a dummy path here

Copy link
Collaborator Author

@eecavanna eecavanna Feb 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't just have a dummy path here

Thanks—I agree. This was a mistake on my part.

Communicating which schema variant we expect, to the caller

The previous default value here would not work because this repository no longer contains a file at ../../../schema/nmdc.schema.json.

My intention by replacing the default value was to give a hint to the user about which variant of the schema we expect them to provide; i.e. the nmdc_materialized_patterns.schema.json variant, not the nmdc.schema.json variant. The code still won't work, though (neither file exists at the path specified).

I'll add a note to the (otherwise empty) docstring, saying that we expect the user to provide the path to the nmdc_materialized_patterns.schema.json variant of the schema when they invoke the function.

The default value

The Runtime repo no longer contains a schema file whose path we could use as the default value.

Invocations of this function

The only invocation of this function I see (in the repo) is the one right below the function's definition; i.e. this one:

if __name__ == "__main__":
    print("study test", test_gold_study_json())

That invocation relies on the function's default values.

The future of this script

Given that this script was already broken (due to the path being nonexistent) before this PR was created...

Do you have a sense for whether anyone uses this script anymore?

It was last updated in 2022 (before I updated the parameter's default value in this PR). I'm guessing someone wrote it to demonstrate something way back when.

Given that it lacked any documentation, maybe they did not expect it to be used by other people (or their future self) or maintained over time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the latest commit (i.e. 2042c09), I removed all the obsolete, hard-coded file paths and replaced them with references to CLI arguments. This way, it is the user's responsibility to tell the script where it can find these files.

Copy link
Collaborator Author

@eecavanna eecavanna Feb 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still like to remove this file from the repo if it's obsolete. That way, we and our successors don't spend time trying to salvage it as the rest of the code base evolves.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @turbomam what should we be using instead of the nmdc_materialized_patterns.schema.json

log_file="study_error.log",
):
r"""
TODO: Document this function.
TODO: Add type hints for this function's parameters and its return value.
"""
valid = validate_json(data_path, schema_path, log_file)

assert valid
Expand Down