Docker penalty

[smathermather]

I don't know if this deserves it's own issue, but I just did a side-by-side of running a big job with and without docker, and I am curious if anyone else has done it.

For the record, with docker it took 10h 25m, without, 10h 5m, yielding about a 3.5% penalty which (short further testing) I attribute to running on docker.

I am super curious if anyone else has done a similar comparison. If so, we might have enough data points to add to the docs.

[coreysnipes]

All of our benchmark tests have been done on docker installs so far, but there's a field in the data for tracking that (VM_TYPE). I'd be interested to see a comparison between host OS types too. My perception is that docker on linux has less overhead, so penalty might be lower with docker-on-ubuntu vs docker-on-win10.

It will be interesting to repeat some of our benchmark tests as well, to get an idea of how much variance we get between runs. Anecdotally my repeat tests vary a little, but tend to fall within a minute or two of each other. Those are mostly smaller (1-3 hr) jobs.

[smathermather]

It's quite possible this difference is mostly noise: maybe I ran top a whole lot more on the first run and taxed the system in that or some other way.