Skip to content

funcX version match requirements between interchange and workers #638

@yadudoc

Description

@yadudoc

PR #637 Removes the enforcement of a strict version match between the interchange and the workers. This version check used to confirm that the components are running the same python version and the same parsl version. While relaxing the first part of the check makes sense given the tests we've run to confirm that this is indeed safe, the second part is still potentially problematic.

  1. The version checks should check the funcx-endpoint versions and not the parsl version.
  2. Assuming that we'd potentially have old containers that would be launched against endpoints, we'll need to make sure that we take care to keep the message protocols consistent here. For eg, changes like the task_cancel Support for cancelling tasks in HighThroughputExecutor #606 needs new messaging between the interchange and the funcx-manager/worker group are breaking changes.

Describe the solution you'd like

  • Replace parsl checks with funcx version checks
  • Plan around how we'd have some safety around potential version mismatches here.
  • We'll definitely need this in CI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions