Flux support

[shahzebsiddiqui]

You'll need to have Flux installed (somewhere) for a user to use a Flux executor of any kind (with reframe, Snakemake, buildtest or other). But you can develop / just try it out with the pre-build containers! If you want to try it out, you can mimic what I did for the tutorial for the Snakemake executor using the Flux sched container:

https://snakemake.readthedocs.io/en/latest/executor_tutorial/flux.html

And specifically you'll want to build a container environment that mimics that: https://github.com/snakemake/snakemake/blob/main/examples/flux/Dockerfile. That's a bit manual approach - I would take it a step further and recommend using a Development Container as an easy way to develop (this is what I do for the Flux Restful API and it starts in VSCode without a hitch!) Once you have an environment with Flux, you can mimic either the design in reframe or the one in snakemake. The Flux python bindings are very easy to use - you import the libraries, create a handle, and then use various functions from flux.job to submit and otherwise interact. So in summary:

• Create a .devcontainer environment to go alongside buildtest, open with VSCode
• Ensure you can start a flux instance (flux start --test-size=4)
• You can then play around with Flux to try it out - here is a good start for that (or just look at the code I linked above for Python).
• Try installing ipython, then running "ipython" and importing flux and flux.job to see if you can submit a job and then interact with it.
• Add your model of an "executor" for flux that uses these Python commands to submit / interact with jobs.

And of course ping the slack / open an issue if you have any questions! Likely the Performance slack #flux channel is the best place for that - let me know if you need an invite. Great you are using Flux!