Command Tutorials to do

[vsoch]

• flux proxy: "I want to connect to a flux instance across clusters with ssh add tutorial to connect to flux between clusters #192
• flux proxy: "send commands to a flux instance you've started" tutorials: add flux proxy command tutorial #200
• flux start: "I want to start my own flux allocation/instance to launch jobs"
• flux mini submit: "I want to run or submit a job to a flux instance" tutorials: add a tutorial on submitting jobs to Flux #194
  • flux mini run tutorials: add flux mini run tutorial/example #202
  • flux job attach tutorials: add pointer to "flux job attach" #204
• flux mini alloc
• flux mini batch - command directives, standard i/o
• flux jobs: - -a, --format, --recursive, --filter, formats
• flux resource list
• flux queue list
• flux queue drain / idle: "waiting for all your jobs to complete"
• flux job cancel/cancelall/flux pkill - tutorials: Add a flux cancel tutorial #210
• flux job kill/killall
• flux job pgrep/pkill
• flux filemap
  Please add to this list as you see fit!

[chu11]

side note on the submitting jobs faster one, as it crosses multiple commands and maybe is more about techniques, may cross the threshold into general "tutorial". I began a skeleton of a tutorial awhile back, not sure if I should divide it up or not.

[vsoch]

@chu11 I think that might fit well for more of an "advanced tutorial" - to submit jobs more efficiently I imagine the user is pretty good at Flux but wants to be able to optimize that. We have entire sections of the docs just for Jobs so maybe as another page there? https://flux-framework.readthedocs.io/en/latest/jobs/index.html

Any part of the site we can also talk about in terms of a total re-organization - I'm not wed to anything and wanting to make it all better!

[grondo]

One thing I've had in the back of my mind is that flux mini bulksubmit is generally inspired by GNU parallel, which has some great tutorials online. So there is a set of users out that aren't HPC users, but want to run things in parallel. Maybe a specific tutorial that is geared toward that class of user would be a nice intro. It could borrow from GNU parallel tutorials, but add on the benefits of having a resident resource manager handling all those jobs instead of a single process:

• oh, you want to quickly list which jobs failed? Here you go (flux jobs -f failed)
• oh, you want to look at the output of a failed job again? flux job attach
• oh, you want to watch output as the jobs execute? flux mini bulksubmit --watch --progress
• oh, you want to cancel something that hasn't run yet? flux jobs + flux job cancel or flux pkill
• oh, you forgot a set of things and want to run them too? (just submit more work!)
• oh, your workload needs more resources? All these examples work whether your Flux instance is running locally on your laptop with flux start -s 1 or on a cluster of 1000 nodes?
• oh, you only have access to a cluster with a traditional resource manager? That's ok, you can start a large Flux instance as a job under many resource managers.

And so on...

[vsoch]

oh, you want to do the Flux thing? Have no fear - Fluxman is here!

I made him for an upcoming (TBA) talk with stable diffusion! He's kind of wonky but that's also why he's great :)

[grondo]

I'm getting a strong Clamps vibe: What you want I should use Flux? The thing I use every day? The thing that I'm named after? You're a freakin' genius!

[vsoch]

@grondo you've out-culture-referenced me and I have to bite - what is Clamps? I might have seen it but I don't remember the name!

[garlick]

They're coming straight toward our proximity! https://www.youtube.com/watch?v=Km_1NMUHjfA

[vsoch]

Omg! How did I miss this one! 😂 yes, clamps! Ty 🙏

[vsoch]

This particular cartoon was also giving me Strong Bad vibes https://youtu.be/90X5NJleYJQ (in blue!)

[cmoussa1]

I can give flux mini submit/run: "I want to run or submit a job to a flux instance" a shot! We can probably leverage the example we have of submitting jobs through CLI and/or API in our flux-workflow-examples repository. It's been a while since this repository has been updated but IMHO it would be nice to insert some of that work that folks have done in the past into our official documentation.

[grondo]

Before we pull any examples from flux-workflow-examples, let's make sure they are up to date and/or correct. Many of those examples were unfortunately out of date already a few years ago.

[wihobbs]

One thing that I noticed today is that we don't have any workflow examples for users running workflows on system-instance Flux clusters (corona, tioga, etc.). We might not want this in flux-docs, since there will be a lot of LC specific questions.

But, some questions that I think users will probably have (because I have had them this week, thanks @cmoussa1 for answering on Slack):

• How do I charge a bank that is not my default bank? (--setattr=system.bank=charge_me)
• Or, how do I change my default bank?
• How do I view the banks I have access to on a given machine? (Slurm: sacctmgr list user whoami witha Flux: flux account view-user whoami)
• What all can I set with --setattr?

These are just things I couldn't find in our docs or by running --help.