Remote docker throws "sh: 1: cargo: not found" error if cargo comes from nix distribution

[david9991]

Checklist

• I've looked through the issues and pull requests for similar reports

Describe your issue

When executing the command DOCKER_HOST=... CROSS_REMOTE=1 ~/.cargo/bin/cross --verbose build --target arm-unknown-linux-gnueabi, I encountered the error sh: 1: cargo: not found.

With verbose output enabled, I discovered that the following command was executed:

/home/gitlab-workspaces/.nix-profile/bin/docker exec cross-stable-x86_64-unknown-linux-gnu-ae81e-79e9716c9-arm-unknown-linux-gnueabi-plcc-blockchain-tools-c1946 sh -c 'mkdir -p '\''/cross/cargo'\'''
/home/gitlab-workspaces/.nix-profile/bin/docker cp -a /home/gitlab-workspaces/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin cross-stable-x86_64-unknown-linux-gnu-ae81e-79e9716c9-arm-unknown-linux-gnueabi-plcc-blockchain-tools-c1946:/cross/rust

The second command copies the /home/gitlab-workspaces/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin directory as /cross/rust instead of copying the folder to /cross/rust/ if the /cross/rust directory does not exist. (Only 'cross/cargo' was created in the previous command)

This behavior will cause the subsequent command:

/home/gitlab-workspaces/.nix-profile/bin/docker exec --user 5001:5001 -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e BROWSER -e 'USER=gitlab-workspaces' -w /project cross-stable-x86_64-unknown-linux-gnu-ae81e-79e9716c9-arm-unknown-linux-gnueabi-plcc-blockchain-tools-c1946 sh -c 'PATH=$PATH:/rust/bin cargo --verbose build --target arm-unknown-linux-gnueabi --target-dir /cross/project/target`

which attempt to use the cargo executable from the /rust/bin folder to fail.

What target(s) are you cross-compiling for?

No response

Which operating system is the host (e.g computer cross is on) running?

• macOS
• Windows
• Linux / BSD
• other OS (specify in description)

What architecture is the host?

• x86_64 / AMD64
• arm32
• arm64 (including Mac M1)

What container engine is cross using?

• docker
• podman
• other container engine (specify in description)

cross version

cross 0.2.5

Example

No response

Additional information / notes

No response

[Emilgardis]

I'm not sure if there was a specific fix for this done, but can you try cross installed from the main branch?

This shouldn't happen, as we create the directories here

[david9991]

@Emilgardis You're right. It's a Nix related issue. the cargo was patched by Nix, so the error message isn't about the cargo itself, but its elf interpreter cannot be found in the container. It can be worked around by setting the interpreter back to /lib64/ld-linux-x86-64.so.2 before being copied from Nix environment.

[Kek5chen]

Yea. I have this issue as well. Sorry to say this but this piece of software that's supposed to be "magic", has never worked for me even once.

Even when using gcc-unwrapped / supplying a version of unwrapped gcc into the container I get linker errors.

The build pipeline shouldn't rely on the host system apart from sharing a kernel.

[Emilgardis]

nix does stuff that's not expected. The expected dependencies for cross is that you have rustup and a rustup installed toolchain. We can't control what nix decides to do. Sorry that you're having issues.

Where should the interpreter live according to the nix modified cargo? We could automatically mount it (unless we already do with the main branch with NIX_STORE)

[Kek5chen]

The funny thing about this is that I DO have a rust up installed tool chain. Managing my rust installation with nix is absolutely horrible for development because i might need different releases or stable or unstable for different software.

I don't manage my rust installation with nix.

[Emilgardis]

Ok, then you have some other issue, this issue is specifically about Nix in CROSS_REMOTE, please create a separate issue or reach out in our matrix (link in readme) if you want help with that @Kek5chen

[Kek5chen]

To be clear, this issue does additionally happen when using a Nix System and I was just confirming that it's still an issue.

I really don't have the time right now and i'm going to just make a vm to compile my program.

I appreciate your support nonetheless.