Operator constraint update: Adding support for idealised calendars (eg. 360 day) #1705
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jfrost-mo
reviewed
Sep 24, 2025
There was a problem hiding this comment.
We want to support the full range of ISO 8601 dates. Additional test cases to add:
- Basic format:
20250924 - Basic format with time:
20250924T0921Z - Month precision:
2025-09 - Microseconds:
2025-09-24T09:21:42.123456 - Alternate UTC representation:
2025-09-24T09:21+00:00 - Indian Standard Time:
2025-09-24T09:21+05:30
We probably don't need to support week dates (2025-W01) or ordinal dates (2025-365).
There was a problem hiding this comment.
There shouldn't be any issue adding a 24 KiB test file. Stick it somewhere under tests/test_data/.
… test.read.py. Modifications: Constraints.py: Commits modifying this file have added the function 'my_pdt_fromstring()' which parses out am ISO date string into temporal coordinates (year, month, day, hour, minute and second) to return a PartialDateTime object to replicate the 'fromisoformat()' method. The function for 'generate_time_constraint()' is also modified such to use 'my_pdt_fromstring()' rather than 'fromisoformat()'. This has therefore allowed for the ability to process files with more idealised calendars included in CF Metadata convention (eg. 360 day). test_constraint.py: A series of test functions have been written to test the 'my_pdt_fromstring()' function added to constraints.py. These verify the function is able to correctly parse a given date string and return a PartialDateTtime. test_read.py: A test function named 'test_read_cubes_generate_time_constraint()' verifies that a cube can be read and correctly constrained along the time coordinate. NOTE: the test function added to test_read.py uses a NetCDF file that has not been included in the pull request. The file in question has a size of 24 KB, similar to other data files included within the test_data folder and uses a 360 day calendar. We would like to further ask for further approval to add this. Fixes #1701
tdh2005
force-pushed
the
operator-constraint-update
branch
from
052b90a to
41e26e1
Compare
|
|
Constraints.py: This commit modifies the file to add wider support for offset formats (e.g. hh, hhmm, hh:mm) with the my_pdt_fromstring() function.
constraints.py: Format of pdt output changed for special case YYYY-MM for the my_pdt_fromstring() function. test_constraints.py: asserts modified for the special case YYYY-MM test to reflect changes to my_pdt_fromstring() function. test_read.py: More specific asserts added to the test_read_cubes_generate_time_constraint() test to reflect file properties.
jfrost-mo
reviewed
Oct 28, 2025
There was a problem hiding this comment.
Sorry for being slow to review this again. Its looking really good, and the test cases are nice and comprehensive.
Because the functionality of creating PartialDateTime objects looks useful, I think it should be moved into _utils.py to make it available to all operators, and renamed to be clearer. The tests will also need moving to test_utils.py
Otherwise it looks ready to go.
src/CSET/operators/constraints.py
Outdated
| Arguments | ||
| --------- | ||
| datestring: str | ||
| ISO date |
There was a problem hiding this comment.
Suggested change
| ISO date | |
| ISO 8601 date. |
src/CSET/operators/constraints.py
Outdated
| time = "" | ||
| elif len(datetime_split) == 2: | ||
| time = datetime_split[1] | ||
|
|
src/CSET/operators/constraints.py
Outdated
| from CSET._common import iter_maybe | ||
|
|
||
|
|
||
| def my_pdt_fromstring( |
There was a problem hiding this comment.
I can see this function being useful for multiple operators. Could it be moved into _utils.py? This also stops it being exposed as a public function via constraints.my_pdt_fromstring.
Furthermore, I think it would be good to rename this function to pdt_fromisoformat, to better describe what it does, and match datetime.fromisoformat.
… test.read.py. Modifications: Constraints.py: Commits modifying this file have added the function 'my_pdt_fromstring()' which parses out am ISO date string into temporal coordinates (year, month, day, hour, minute and second) to return a PartialDateTime object to replicate the 'fromisoformat()' method. The function for 'generate_time_constraint()' is also modified such to use 'my_pdt_fromstring()' rather than 'fromisoformat()'. This has therefore allowed for the ability to process files with more idealised calendars included in CF Metadata convention (eg. 360 day). test_constraint.py: A series of test functions have been written to test the 'my_pdt_fromstring()' function added to constraints.py. These verify the function is able to correctly parse a given date string and return a PartialDateTtime. test_read.py: A test function named 'test_read_cubes_generate_time_constraint()' verifies that a cube can be read and correctly constrained along the time coordinate. NOTE: the test function added to test_read.py uses a NetCDF file that has not been included in the pull request. The file in question has a size of 24 KB, similar to other data files included within the test_data folder and uses a 360 day calendar. We would like to further ask for further approval to add this. Fixes #1701
Constraints.py: This commit modifies the file to add wider support for offset formats (e.g. hh, hhmm, hh:mm) with the my_pdt_fromstring() function.
constraints.py: Format of pdt output changed for special case YYYY-MM for the my_pdt_fromstring() function. test_constraints.py: asserts modified for the special case YYYY-MM test to reflect changes to my_pdt_fromstring() function. test_read.py: More specific asserts added to the test_read_cubes_generate_time_constraint() test to reflect file properties.
tdh2005
force-pushed
the
operator-constraint-update
branch
from
3337295 to
5f6c414
Compare
…fice/CSET into operator-constraint-update Updating branch with changes to main branch.
- my_pdt_fromstring() : This function has been renamed to pdt_fromisoformat() such to allign to what it replaces (.fromisoformat()). - _utils.py : The newly named pdt_fromisoformat() function has been moved from constraints.py into utils.py, such that it can be used in conjunction to multiple different operators. - test_utils.py : The tests made for the pdt_fromisoformat() function have been moved to test_utils.py, to reflect the relocation of the function from constraints.py to _utils.py. - test_read.py : This file has been modified to handle the changes to name and location of the pdt_fromisoformat() function.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1701
This update modifies 3 files; constraints.py, test_constraints.py and test.read.py.
Modifications:
Constraints.py: Commits modifying this file have added the function 'my_pdt_fromstring()' which parses out am ISO date string into temporal coordinates (year, month, day, hour, minute and second) to return a PartialDateTime object to replicate the 'fromisoformat()' method. The function for 'generate_time_constraint()' is also modified such to use 'my_pdt_fromstring()' rather than 'fromisoformat()'. This has therefore allowed for the ability to process files with more idealised calendars included in CF Metadata convention (eg. 360 day).
test_constraint.py: A series of test functions have been written to test the 'my_pdt_fromstring()' function added to constraints.py. These verify the function is able to correctly parse a given date string and return a PartialDateTtime.
test_read.py: A test function named 'test_read_cubes_generate_time_constraint()' verifies that a cube can be read and correctly constrained along the time coordinate.
NOTE: the test function added to test_read.py uses a NetCDF file that has not been included in the pull request. The file in question has a size of 24 KB, similar to other data files included within the test_data folder and uses a 360 day calendar. We would like to further ask for further approval to add this.