Ability to set `weight` on a per-`Task` basis?

[dotsdl]

Perhaps in addition to priority, setting a weight on a Task for a given Transformation could be used to encode the difficulty of the Transformation (e.g. how much sampling is required to get a converged result). This would help with resource allocation.

Under this scheme, priority would function as a partitioning scheme, with Tasks of equal priority selected based on their weight. It's also worth considering if priority should be deprecated in favor of weight to avoid too much complexity here. This would be a simplification, making weight the only sole selection scheme at both the TaskQueue and Task level.

[dotsdl]

Instead of having TaskQueues, we could switch to each AlchemicalNetwork having a TaskHub. Tasks would then be associated with the TaskHub, and the connecting edge could feature a weight property. Tasks would then be drawn from a TaskHub based on their weight, just as TaskHubs themselves are selected based on their own weight property.

This would ditch determinism or any sense of order for Tasks in terms of execution. We could perhaps retain priority as a Task property that takes precedence over weight considerations (Tasks of equal, highest priority for a TaskHub would be selected for compute based on weight). We could also consider a tag property for Tasks that would allow compute services to filter for only Tasks with a given set of tags.

[dotsdl]

@jchodera do you have a preference? @dwhswenson, @richardjgowers any thoughts on this?

Dropping the linked-list topology that TaskQueues currently have in favor of TaskHubs (think hub-and-spokes, where Tasks are on the spokes) would be a simplification, so might be worth pursuing fairly soon.

[dotsdl]

I need to check if Cypher exposes any random choice functions so we can do our weighted selection and claiming of Tasks in a single transaction.

[jchodera]

I very much support the idea of using weights (rather than priorities) for both Task and TaskHub. This enables the execution engine to be more relaxed in how it uses these weights to stochastically prioritize effort, ensures we don't have to worry about enforcing strict priorities, and will support adaptive weight updating strategies that are only weakly coupled to the execution engine.

[dotsdl]

Adding this to release 0.2.0; we'll proceed with TaskQueues for now for 0.1.0, but switch to TaskHubs shortly after.

There does not appear to be a way to directly do a weighted random choice in Cypher, nor do a cumulative sum to implement one in a query. So we may have to resort to doing the weighted selection in Python within the compute API's Neo4jStore. I believe we can make this happen within a single transaction even if it isn't all in a single query.

[jchodera]

I wonder if there is a simple way to do this with rand() using a two-pass approach:

1. Retrieve all the nodes that match the desired criteria and sum up their weights
2. Make another query using a rand() selection designed to return everything where rand() <= weight * number_of_nodes_desired / weight_sum, which should produce approximately number_of_nodes_desired results

[hmacdope]

@dotsdl and I have decided to split this into two seperate issues. Redesigning the TaskQueue system to use TaskHubs is now in issue #78 while adding "compute tags" that allow tasks to be called by some kind of identifier is now in issue #79

[hmacdope]

This was mostly addressed by #80. Now that we have a separate issue for "tags" in #79 I am going to go ahead and close @dotsdl. Feel free to re-open if you want.