Lazy compilation for numba gufuncs #103
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sallen12
reviewed
Aug 4, 2025
| "void(float64[:], float64[:,:], float64[:])", | ||
| ], | ||
| "(d),(m,d)->()", | ||
| cache=True, |
Collaborator
There was a problem hiding this comment.
Don't fully understand the motivation for caching the energy score gufunc but not the other gufuncs imported at import time, e.g. (fair and nrg) crps and variogram score?
The weighted interval score is also cached but lazily loaded. Again, why this score and not the other scores?
sallen12
approved these changes
Aug 4, 2025
Collaborator
sallen12
left a comment
sallen12
left a comment
There was a problem hiding this comment.
Looks good, thanks! I have made one comment on caching of the gufuncs (why particular gufuncs have been cached and not others), but it is more for my understanding rather than something that needs fixing.
sallen12
reviewed
Aug 4, 2025
| "nrg": _crps_ensemble_nrg_gufunc, | ||
| "pwm": _crps_ensemble_pwm_gufunc, | ||
| "int": lazy_gufunc_wrapper_uv(_crps_ensemble_int_gufunc), | ||
| "nrg": lazy_gufunc_wrapper_uv(_crps_ensemble_nrg_gufunc), |
Collaborator
There was a problem hiding this comment.
nrg estimator could also be lazily loaded - currently it also has the longer decorator, as if it is not lazily loaded
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactors numba gufuncs to be lazily/dynamically compiled, with the exception of a few common functions which will still be compiled eagerly during import. This brings a significant improvement in terms of import time: on my (fairly slow) machine, it goes from 19s to 3s. Closes #80.