Replication package for A Comparative Study on the Suitability of Cloud Simulators for Estimating Energy Usage of Microservices.
To replicate the results presented in this study, the following steps can be followed:
Ground truth data collection based on: Berta Rodriguez Sanchez. 2024. Multivariate Anomaly Detection and Root Cause Analysis of Energy Issues in Microservice-based Systems. Master’s thesis. University of Amsterdam and Vrije Universiteit Amsterdam, Amsterdam. https://github.com/bertars/Thesis.
The systems used for this study are SockShop and TrainTicket. ExperimentRunner is used to automate the data collection.
For how run the ground truth data collection, see the ReadMe.
- Trace selection: To select the repetitions closest to the mean behaviour for each treatment, run select_traces.py.
- Extract idle power values: To extract the idle power values from the ground truth, run get_idle_power.py. The max power values are extracted in step 4.
- To get the IPC values for sockshop and trainticket, run get_ipc.sh on the server running the microservice system.
- Trace generation: To generate the traces, run gen_tracs.py.
All simulations can be carried out by running the run_all.sh script, which in turn runs each simulator's run script, located in their respective folder in the simulators directory.
The simulators under test are as follows:
- CloudSim/NetworkCloudSim v7.0.0 (simulators/cloudsim)
- DISSECT-CF ((simulators/dissectcf))
- OpenDC v2.4d (simulators/openDC)
- SimGrid v4.0 ((simulators/simgrid))
The scripts in the evaluation folder can be used to replicate the analysis presented in the study.
In order to do so, first run gen_energy.py and gen_sim_energy.py. Because of the large number of measures, the dataset has to be split into different csv files, which are too big to upload to GitHub.
Then, evaluation/get_stats.ipynb and evaluation/runtime/get_runtimes.py can be run to carry out the analysis.