You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Avoid loading samples for each iteration in variables loop inside run.sensitivity.analysis #3859
While reviewing the implementation of run.sensitivity.analysis, I noticed that samples.Rdata and sensitivity.samples.Rdata are loaded inside the loop over variables and this means the files are read from disk on every iteration:
Since these files do not depend on variable, they appear to be identical for each iteration, the current implementation performs repeated disk I/O that could be avoided.
I thnk Instead of loading the files during every iteration, they could be loaded once before the loop and reused during the iterations, also I think we have 2 options to refactor this:
First solution: Load the .Rdata files once before the loop using load_local() function which converts the loaded environment to a list, then Inside the loop samples <- samples_base
Since lists in R use copy-on-modify semantics, a copy would only occur when modifications are made.
Second solution: we could keep the current environment structure but load the files only once,
keep the same load logic before the loop
Then inside the loop create a fresh environment copy for each iterationsamples <- rlang::env_clone(samples_base)
I think both approaches would avoid repeated disk reads overhead inside the loop.
But Before proposing a change, I wanted to ask is there a reason the files are reloaded on every iteration?
I can't remember any design reason why we were loading samples repeatedly within a site. Across sites, you may need to load separate samples if the different sites have different parameter matrices / PFTs.
Another thing I've previously noted is that with the addition of the input_design matrix, which only occurred last year, I think run.sensitivity.analysis can probably be deprecated as sensitivity analyses should now just be an ensemble analysis with a specific input_design and a specific parameter matrix.
@mdietze Thanks, I think this fits well with the “increase modularity” GSoC refactor, that makes sense deprecating run.sensitivity.analysis as a runner now that input_design + explicit samples define the runs, but do you think the same idea applies to write.sa.configs? With input_design and the required samples through run.write.configs, it feels like SA config writing could be treated as a special case of the ensemble pipeline.
Activity
mdietze commentedon Mar 10, 2026
I can't remember any design reason why we were loading samples repeatedly within a site. Across sites, you may need to load separate samples if the different sites have different parameter matrices / PFTs.
Another thing I've previously noted is that with the addition of the input_design matrix, which only occurred last year, I think run.sensitivity.analysis can probably be deprecated as sensitivity analyses should now just be an ensemble analysis with a specific input_design and a specific parameter matrix.
Thomas-Sedhom commentedon Mar 10, 2026
@mdietze Thanks, I think this fits well with the “increase modularity” GSoC refactor, that makes sense deprecating run.sensitivity.analysis as a runner now that input_design + explicit samples define the runs, but do you think the same idea applies to write.sa.configs? With input_design and the required samples through run.write.configs, it feels like SA config writing could be treated as a special case of the ensemble pipeline.