Develop optimal approach for running SDA simulations to maximize I/O speeds but reduce storage costs with s3
What is the best option for fast I/O during large simulations across the nodes to optimize simulation speeds during execution?
Can we develop post-processing code that triggers after completion of each simulation set that moves the outputs to s3?
Are there decent R packages for accessing data on s3 directly like python?
Other considerations?
Develop optimal approach for running SDA simulations to maximize I/O speeds but reduce storage costs with s3
What is the best option for fast I/O during large simulations across the nodes to optimize simulation speeds during execution?
Can we develop post-processing code that triggers after completion of each simulation set that moves the outputs to s3?
Are there decent R packages for accessing data on s3 directly like python?
Other considerations?