Where should the data processing actually run?

[mccalluc]

Having a webapplication spawn a subprocess, much less a docker container, is a bad idea. Even apart from security concerns, there's a mismatch between the needs of the webserver (minimal! except for storage... see #19 and #20.) and the needs of the processing (huge, above 10G RAM, at least.)

This might be appropriate for AWS Lambda, if the jobs can run quickly enough. Google or Azure might have their own offerings in this space.

Different compute platforms would support different storage possibilities.