Skip to content

Remote execution architecture woes #2825

Open
@peterebden

Description

@peterebden

This is a little bit of a stress test / awkward case, which didn't go so well. Test setup:
Machine 1, linux_amd64, running a set of remote execution servers
Machine 2, linux_arm64, the client.

Building stuff fails approximately immediately with an exec format error from arcat (which just happens to be the most proximate thing, there's nothing specifically up with it).

Things seem to generally be set up correctly. What happens is:

  • Please sees arcat is a tool, and so will always build it for the host arch (which is correct for local execution)
  • It asks the remote execution server to do the same, which it does, duly producing arcat-linux-arm64
  • It now attaches that artifact to another action, which fails on the remote executor which is linux_amd64

I'm not really certain what is meant to happen here. There are Platform messages which we should maybe be doing more with - they are currently just a config setting but we should probably automatically attach the arch / ISA which could/would/should at least in this case have the server reject the action saying that it can't fulfil the requested architecture.
The arbitrarily nice situation is that Please finds out the server is on a different architecture and swaps itself to match that (essentially pretending to be linux_amd64 and cross-compiling to linux_arm64). I haven't thought much about that but I just bet it's gonna be hard :( I also don't think that the rex protocol has any way for the client to find out about the server this way but it seems reasonable for this to be specified in the config (presumably you know something about the remote execution setup and it's not a total black box).

This might also be fairly doomed for interactions with local = True which would also imply an architecture switch which we're not really set up for. Arguably this would all work more nicely if 'local' and 'remote' were architectures that could be arbitrarily reconfigured.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions