Add linkage attributes to extern "C" blocks

[saethlin]

This crate is now used in the compiler since rust-lang/rust#126930.
The problem I'm fixing is just like rust-lang/rust#118084; if you try to run

RUSTFLAGS=-Zcross-crate-inline-threshold=always ./x.py build library

After around 30 minutes (-Zcross-crate-inline-threshold=always makes compilation much slower) you get

  = note: rust-lld: error: undefined symbol: blake3_hash_many_avx512
          >>> referenced by ffi_avx512.rs:49 (/home/ben/.cargo/registry/src/index.crates.io-6f17d22bba15001f/blake3-1.5.2/src/ffi_avx512.rs:49)
          >>>               /home/ben/rust-master/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_main-1766fdd0f895a9f6.rustc_main.5f5b835cb13221b6-cgu.0.rcgu.o:(blake3::compress_subtree_wide::<blake3::join::SerialJoin>)
          >>> referenced by ffi_avx512.rs:49 (/home/ben/.cargo/registry/src/index.crates.io-6f17d22bba15001f/blake3-1.5.2/src/ffi_avx512.rs:49)
          >>>               /home/ben/rust-master/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_main-1766fdd0f895a9f6.rustc_main.5f5b835cb13221b6-cgu.0.rcgu.o:(blake3::compress_parents_parallel)
... any many more!

I think this is exactly the same problem as the above-linked issue: the blake3 crate does not have any link(name = attributes, so it only links correctly if all calls to the extern "C" functions are monomorphized by blake3 crate. But with -Zcross-crate-inline-threshold=always, nearly all monomorphization is done by the last crate to be compiled, and without these attributes we lose track of these symbols by the time we need them.

I'm not filing an issue because I'm pretty sure nobody else cares about my silly flag continuing to work when bootstrapping the compiler, but I occasionally find legitimate codegen bugs with it, so I'm sending you the patch to keep it working.

I've confirmed in my local dev setup that this PR makes it again possible to build the compiler with -Zcross-crate-inline-threshold=always.

[oconnor663]

Interesting, yes, let's fix this. CI is showing some build failures in cargo test --features=prefer_intrinsics. As you'll see, we get these symbols from different places in different modes. One confusing point is that SSE and AVX2 are available in stable Rust, so we have pure Rust intrinsics implementations for those, but not for AVX-512, which still links against C code (but different C sources). Can you get that working on your box?

[saethlin]

Oh I see what you mean, the avx512 situation adds another dimension to the matrix. I'll need a bit to untangle this.

[saethlin]

Oh dear it's the workflow approval dance.

[saethlin]

A bit of scope creep later, now I've:

• gated all of the linkage attributes on their matching cfg that's set by the build script
• renamed blake3_neon to blake3_neon_ffi so that it matches
• Added two new cfgs so that the build script can indicate which library is providing avx512 implementations

[saethlin]

Works on my machine too!

[saethlin]

👋 Just generating a notification to make sure this PR isn't lost.

[oconnor663]

Apologies for letting this sit for so long. I'll get back to it this weekend.

renamed blake3_neon to blake3_neon_ffi so that it matches

I think I added blake3_neon in a few more places since your last rebase, so that's probably causing some warnings that become errors in CI.

[oconnor663]

All of these extensions have assembly implementations and intrinsics implementations, which we switch between with --feature prefer_intrinsics. Do they all need the same treatment as AVX-512?

More broadly, could you help me understand what the link(...) attribute actually means :)

[saethlin]

I started digging into this and actually now I have questions about why the compiler seems to correctly forward native libraries dependencies through the crate graph if they come from #[link( but not if they come from command line arguments. I've poked some other compiler people, and I'll try to get back to you soon.