Can sim.save() and sim.load() save and restore the whole Sim object?

[lampk]

I am trying to see whether I can run the simulation using starsim in python, save the whole simulation into a file using sim.save() and then use R to work with the simulation output. However, it seems that using sim.save() and sim.load() are not able to restore the whole simulation in Python (my example below). Do I miss something when using sim.save() and sim.load()? Is there anyway to export the whole starsim simulation into a file and then restore it in another python session or extract its components using other software, eg R?

I did a simple simulation and saved it into file results:

import starsim as ss
sim = ss.Sim(diseases='sir', networks='random')
sim.run()
sim.save('results')

However, when I opened a new jupyter notebook and tried to load the file results, I could not access the simulation results. The error message suggests that I have to re-run the simulation again:

import starsim as ss
sim = ss.Sim(diseases='sir', networks='random')
sim.load('results')
sim.results

Results(sim)

sim.plot()

---

RuntimeError Traceback (most recent call last)
Cell In[8], line 1
----> 1 sim.plot()

File /opt/anaconda3/lib/python3.11/site-packages/starsim/sim.py:563, in Sim.plot(self, key, fig, style, show_data, show_skipped, show_module, show_label, fig_kw, plot_kw, scatter_kw)
546 def plot(self, key=None, fig=None, style='fancy', show_data=True, show_skipped=False,
547 show_module=26, show_label=False, fig_kw=None, plot_kw=None, scatter_kw=None):
548 """
549 Plot all results in the Sim object
550
(...)
561 scatter_kw (dict): passed to plt.scatter(), for plotting the data
562 """
--> 563 self.check_results_ready('Please run the sim before plotting')
565 # Configuration
566 flat = self.results.flatten()

File /opt/anaconda3/lib/python3.11/site-packages/starsim/sim.py:451, in Sim.check_results_ready(self, errormsg)
449 errormsg = 'Please run the sim first'
450 if not self.results_ready: # pragma: no cover
--> 451 raise RuntimeError(errormsg)
452 return

RuntimeError: Please run the sim before plotting

[cliffckerr]

Hi @lampk , thanks for your inquiry! The issue is that Sim.load is a static method, which means that it creates a new Sim instance rather than modifying the existing one. If you assign Sim.load() to a new variable, then it works, e.g.:

import starsim as ss

# Run and save the sim
sim = ss.Sim(diseases='sir', networks='random')
sim.run()
sim.save('sim.obj')

# Load and plot the sim
sim2 = ss.Sim.load('sim.obj')
sim2.plot()

(Note that by default this saves a "shrunken" version of the sim, which includes the results but not all the details of all the agents. To save the complete sim exactly as-is, for example if you want to save partway through a run and then restart the run, use sim.save(shrink=False).)

However, I agree this is confusing behavior, so I've created an issue here to change it from ss.Sim.load() to simply ss.load():
#766

[lampk]

Thank you very much, Cliff, for your instant response. Now I understand and can save and load the simulation in Python.

Just a follow-up question: Is it currently possible to read the saved simulation file using R? My understanding is that the saved file is in pickle format; therefore I tried to use reticulate and read_pickle but did not succeed...

library(reticulate)
pd <- import("pandas")
ss <- import("starsim")
starsim_sim <- pd$read_pickle(file.path("sim.obj"))

Error in py_call_impl(callable, call_args$unnamed, call_args$named) :
ModuleNotFoundError: No module named 'starsim.sim'
Run reticulate::py_last_error() for details.

[cliffckerr]

The reason is that pd$read_pickle is only for pandas pickles. But the good news is that all of this works in R -- not just loading the sim!

library(reticulate)
ss <- import("starsim")

# Run and save the sim
sim <- ss$Sim(diseases='sir', networks='random')
sim$run()
sim$save('sim.obj')

# Load the sim
sim2 <- ss$Sim$load('sim.obj')

# Convert the sim results to a dataframe
df <- sim2$to_df()

(If you just replace starsim_sim <- pd$read_pickle(file.path("sim.obj")) in your code with starsim_sim <- ss$Sim$load('sim.obj'), then that should work too.)

The one thing that we're still working on is getting plotting to work (i.e. sim$plot()). However, once you export the results as a dataframe, of course you can use any standard R plotting library (e.g. ggplot).

I also wanted to mention that we're working on a Starsim R package, and we hope to have a first version ready within a week.