Add funtionality to save named lists of plans from the queue or queue history and restore them later

[whs92]

Users often want to run plans or sequences of plans they have run before. For example, they might have a series of plans that are always run as a survey of a sample, or to look at a particular feature etc.

If the plans are run often enough you could make a new plan containing them and then add that to the RE environment. For example:

def survey_scan():

    yield from xas(....
    yield from xas(.....
    yield from change_edge(...
    yield from xps(.....
    yield from xps(.....

However this involves some programming. You also loose the ability to reorder or modify the plans within this plan survey_scan.

If instead it was possible to define and store named lists of plans which could be restored to the queue, then users of graphical interfaces to the queuserver could create their favorite series of plans by submitting things to the queue and then name them to be searched and restored later.

I think we could add the following commands to the re_manager_api.

method	batch_database_item_add
description	Adds a named batch of plans to a redis database
parameters	items: list, the list containing a batch of items. Each element is a dictionary containing valid set of item parameters (see instructions for queue_item_add API). An empty item list will also be accepted. name:str, the name to give this batch. Must be unique user_group: str, the name of the user group (e.g. ‘primary’). user: str,the name of the user (e.g. ‘Default User’). The name is included in the item metadata and may be used to identify the user who added the item to the database.
returns	success: boolean, successful if added. Fails if there is already a batch with this name in the database
execution	immediate

method	batch_database_item_remove
description	removes a named batch of plans from redis database
parameters	name:str, the name to give this batch. Must be unique
returns	success: boolean, successful if removed. Fails if no batch exists with this name
execution	immediate

method	batch_database_item_rename
description	renames a named batch of plans stored on a redis database
parameters	old_name:str, the name to replace name:str, the name to give this batch. Must be unique user_group: str, the name of the user group (e.g. ‘primary’). user: str,the name of the user (e.g. ‘Default User’). The name is included in the item metadata and may be used to identify the user who named the item.
returns	success: boolean, successful if added. Fails if there is already a batch with this name in the database
execution	immediate

method	batch_database_clear
description	removes all items in the database of batches of plans
parameters
returns	success: boolean, successful if cleared. Fails if ?
execution	immediate

method	batch_database_batches
description	returns a dict containing keys which are the batch names and entries which are lists containing batches of items.
parameters
returns	success: boolean, successful if received batches: dict of lists where each element in the list is a dictionary containing valid set of item parameters (see instructions for queue_item_add API).
execution	immediate

This problem could be addressed a number of different ways, but I think putting it on the queueserver is beneficial because multiple clients might want to access what a particular user has submitted. Additionally if the client application stops, the lists of plans is not lost.

[cjtitus]

In this idea, when you restore a named batch of runs to the queue, it would be functionally identical to adding a series of queue items one-by-one, correct?

Could this be expanded to the following ideas:

• a batch_uid/batch_name that would link related queue items together
• groups of scans that are "glued" together, and can only be added/removed/reordered as a block

Or do you see these as being implemented in a totally different fashion, at a different time?

I do think the proposal presented is elegantly simple, and useful as-is.

[whs92]

@cjtitus

In this idea, when you restore a named batch of runs to the queue, it would be functionally identical to adding a series of queue items one-by-one, correct?

Yes that was the intention.

I think adding by default a batch_name and a batch_uid is a good idea and let's us keep track of things.

I think batches of plans could be glued together if the client gui wanted to do that. I would not bake that in to the server though.

Thanks for your feedback!

[whs92]

P.S. we have discussed a name for this group/series/collection/queue of plans in the last months. I happened on "batch" because it was already used in the queueserver

[tacaswell]

Given that there is already an atomic add/remove API, can the be proto-typed fully client side first and then added to the server later? I suspect the exact data structures and atomic API that is needed will take some iteration to get right. At a minimum "dump to a yaml file in $HOME/.cache" is good MVP path.

a batch_uid/batch_name that would link related queue items together

It is easy for the client to inject this as metadata into all of the runs it adds

groups of scans that are "glued" together, and can only be added/removed/reordered as a block

This is the same as writing a plan that does them all. There is bp.pchain which may be worth special casing to allow a list of plans to be (with no flow control) concatenated at the QS level (or the serialization may "just work"?).