Skip to content

Implement distributed tasks #7088

@viluon

Description

@viluon

For example, distcc. The idea is simple: if I have lots of CPUs on a remote machine, I can take a task's inputs along with its source code, upload those to the remote machine, let it compile the task and run it to produce the outputs, then download the results.

In reality, that requires all the inputs and the "source code" to be at least serializable (plus all the other steps needed to make this actually efficient). And it's probably wrong to send over a single task if there's a larger subgraph of the task graph with a narrow, serializable cut -- i.e. two tasks may not have serializable inputs, but extend that to three tasks and they might.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions