Skip to content

Add compact, sortable 'hashing' strategy for step ids/workspaces#465

Merged
jwhite242 merged 12 commits intodevelopfrom
feature/sortable_workspace_hash
Feb 4, 2026
Merged

Add compact, sortable 'hashing' strategy for step ids/workspaces#465
jwhite242 merged 12 commits intodevelopfrom
feature/sortable_workspace_hash

Conversation

@jwhite242
Copy link
Copy Markdown
Collaborator

Implements MEP 003's proposal for more human readable hashing of workspaces/step id's. Replaces default 'md5' algorithm for hashing step combinations in favor of sortable/human readable 'hash'. This is not a true 'hash', in that it needs all combos for a step to compute the pattern. Pattern is: step_name_instance_<number>, where is a zero padded integer (zero based index) based on the number of used step combinations for a particular step. Zero padding ensures proper sorting on filesystems to improve usability.

  • replaces original md5 hash algorithm
  • reuses --hashws cli option to activate
  • adds local integration test
  • adds test fixtures for checking workspace layouts relative to study root (can't hardcode absolute paths in tests)
  • adds test fixtures for checking substituted workspace tokens in generated step scripts.

Copy link
Copy Markdown
Collaborator

@doutriaux1 doutriaux1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, only suggestion I have that can be ignored or added in a later PR would be to allow to control the hashing from within the yaml file either via key or env variable.

pytest.mark.integration,]


# @pytest.mark.parametrize(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this commented out on purpose?

Demonstrates how to display a tree of files / directories with the Tree renderable.
"""

# import os
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clean this up?

)

# Dummy test to force failure and show outputs
# assert completed_successfully == 'NOT A REAL STATUS'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have to do that if you want to see any of the printout's: pytest eats everything unless there's a failure. Really need some alternative switch in here that doesn't leave this kind of debris behind for a nicer test building/debugging workflow... But not on this pr, so will delete all these debug outputs

@jwhite242
Copy link
Copy Markdown
Collaborator Author

Looks good, only suggestion I have that can be ignored or added in a later PR would be to allow to control the hashing from within the yaml file either via key or env variable.

Yeah, that's gonna be a whole other pr. There's currently no where to put any of the study config in spec and I need to explore some alternatives to cluttering up the env/variables block with it as things like that + autoyes/rlimit/sleep have a different purpose from variables/labels/etc

@jwhite242 jwhite242 merged commit cef9934 into develop Feb 4, 2026
14 checks passed
jwhite242 added a commit that referenced this pull request Mar 26, 2026
* Extra Flux Options  (#459)
* Add autoyes to cancel command  (#461)
* Add compact, sortable 'hashing' strategy for step ids/workspaces (#465)
* Update flux exclusive behavior modify task info passed to allocation/launcher (#467)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants