Currently, the Appion controllers will loop indefinitely until they are killed (i.e., their Slurm batch job is cancelled or killed by the scheduler due to hitting max walltime). They loop regardless of whether data acquisition is finished and end up wasting cycles / taking up cluster resources unnecessarily.
Ideally, there would be some way for each instrument (i.e., microscope, camera) to keep track of which session is currently actively using the instrument, and we could perform some check to see if the session that is being processed is no longer active. If the session that's being processed is no longer active and all images are processed for a given run, the controller could then exit and free up CPUs.
Note that the "all images are processed" criterion is necessary because microscope operators may want to run motion correction on already collected data with different parameters (or they may not run motion correction / ctf estimation until after data collection has finished).
Currently, the Appion controllers will loop indefinitely until they are killed (i.e., their Slurm batch job is cancelled or killed by the scheduler due to hitting max walltime). They loop regardless of whether data acquisition is finished and end up wasting cycles / taking up cluster resources unnecessarily.
Ideally, there would be some way for each instrument (i.e., microscope, camera) to keep track of which session is currently actively using the instrument, and we could perform some check to see if the session that is being processed is no longer active. If the session that's being processed is no longer active and all images are processed for a given run, the controller could then exit and free up CPUs.
Note that the "all images are processed" criterion is necessary because microscope operators may want to run motion correction on already collected data with different parameters (or they may not run motion correction / ctf estimation until after data collection has finished).