Skip to content

Add exec and invocation durations#4372

Merged
jmsmkn merged 6 commits into
mainfrom
upgrade_sagemaker_shim_6
Oct 22, 2025
Merged

Add exec and invocation durations#4372
jmsmkn merged 6 commits into
mainfrom
upgrade_sagemaker_shim_6

Conversation

@jmsmkn
Copy link
Copy Markdown
Member

@jmsmkn jmsmkn commented Oct 22, 2025

Adds two new fields:

  • exec_duration
    • The duration of the execution, if measured. Excludes data validation, container pulling, model downloading, data downloading and data uploading times. Includes model loading time, input data loading time, processing time, output data writing time and any delays from shared hardware issues.
  • invoke_duration
    • The duration of the execution, if measured. Excludes data validation, container pulling, model downloading, data downloading and data uploading times. Potentially excludes model loading time, depending on the users implementation. Includes input data loading time, processing time, output data writing time and any delays from shared hardware issues.
    • This will not be set for now and is for future use, it is there for challenge admins to get used to the fact that there are two fields.

One, both or neither will be set. Neither will be set for existing jobs.

Setting and serializing these values will follow in a future PR after all of the container images are upgraded.

Note: There is also billing_duration which includes everything, but we might have to change the implementation of that if we go to batch jobs so this will not be serialized for now.

See https://github.com/DIAGNijmegen/rse-roadmap/issues/441

@jmsmkn jmsmkn changed the title Upgrade sagemaker shim 6 Add exec and invocation durations Oct 22, 2025
@jmsmkn jmsmkn requested a review from koopmant October 22, 2025 13:26
@jmsmkn jmsmkn marked this pull request as ready for review October 22, 2025 13:27
@jmsmkn jmsmkn requested a review from amickan as a code owner October 22, 2025 13:27
Comment thread app/grandchallenge/components/backends/base.py
Copy link
Copy Markdown
Contributor

@koopmant koopmant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@koopmant koopmant removed their assignment Oct 22, 2025
@jmsmkn jmsmkn merged commit 3993ac2 into main Oct 22, 2025
9 of 10 checks passed
@jmsmkn jmsmkn deleted the upgrade_sagemaker_shim_6 branch October 22, 2025 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants