Rewrite AArch64 outlined atomic functions with asm

[pmur]

These functions should follow aapcs, so a strictly asm version shouldn't be needed. This resolves some of the ugliness of reading HAVE_LSE_ATOMICS and enabling bti on various targets.

When compiled with opt-level=1 or higher, these produce nearly identical asm on linux with minor changes to register choices loading HAVE_LSE_ATOMICS.

Some minor macro changes are made to ensure existing lse tests continue to build and run.

TODO: collate and post asm differences. They should be minor.

TODO: figure out why tests now need --features unmangled-names to build now.

Questions: Do the test suites also run with opt-level=1? This should work at 0 (it did for my testing).

[tgross35]

When compiled with opt-level=1 or higher, these produce nearly identical asm on linux with minor changes to register choices loading HAVE_LSE_ATOMICS.

TODO: figure out why tests now need --features unmangled-names to build now.

naked functions are special and always get a public module https://github.com/pmur/compiler-builtins/blob/6ad91ea6dbae4f2d0ce36989e0a2874c43d8f025/compiler-builtins/src/macros.rs#L378-L401. I think you could fix tests by removing one ::$name from this line

compiler-builtins/builtins-test/tests/lse.rs

Line 135 in 9e64861

assert_eq!(old, unsafe { compiler_builtins::aarch64_outline_atomics::$name::$name(val, &mut target) }, "{} should return original value", stringify!($name));

(please rewrap it if you do, don't know how I missed that overflow).

Questions: Do the test suites also run with opt-level=1? This should work at 0 (it did for my testing).

We run with a few options, unoptimized and opt-level=3 should both be covered. Any reason for opt-level=1 specifically?

compiler-builtins/ci/run.sh

Lines 67 to 73 in 9e64861

	asgroup "${test_builtins[@]}"
	asgroup "${test_builtins[@]}" --release
	asgroup "${test_builtins[@]}" --features c
	asgroup "${test_builtins[@]}" --features c --release
	asgroup "${test_builtins[@]}" --benches --release
	asgroup "${test_builtins[@]}" --no-default-features
	asgroup "${test_builtins[@]}" --no-default-features --release

View changes since this review

[tgross35]

The param can just be made mut expected: int_ty!($bytes) (applies a few places)

[pmur]

That currently runs afoul of intrinsics!. I am not sure how (or if) I can teach it how to accept a mut arg.

[pmur]

We run with a few options, unoptimized and opt-level=3 should both be covered. Any reason for opt-level=1 specifically?

No, only that it is where the codegen becomes comparable to the previous implementation. Thanks for the incredible fast code review 😮 .

[pmur]

Attaching a diff of the naked asm vs native asm implementation with optimization enabled.

It's ~1300 lines like:

-   0:  90000010        adrp    x16, 0 <compiler_builtins::aarch64_outline_atomics::HAVE_LSE_ATOMICS>
-   4:  39400210        ldrb    w16, [x16]
-   8:  34000070        cbz     w16, 14 <__aarch64_cas16_acq_rel+0x14>
+   0:  90000008        adrp    x8, 0 <compiler_builtins::aarch64_outline_atomics::HAVE_LSE_ATOMICS>
+   4:  39400108        ldrb    w8, [x8]
+   8:  34000068        cbz     w8, 14 <__aarch64_cas16_acq_rel+0x14>
    c:  4860fc82        caspal  x0, x1, x2, x3, [x4]
   10:  d65f03c0        ret
   14:  aa0003f0        mov     x16, x0

Notes about methodology:

# Extract functions into separate dirs and files
aarch64-linux-gnu-objdump -dC rlib > rlib.txt
gawk 'BEGIN { ON=0 } match($0, /<(__aarch64.*)>:/, name) { ON = 1; NAME=name[1] ".txt"; print NAME } /^$/ { ON = 0 } ON == 1 {print $0 >> NAME }' rlib.txt
# Diff old and new
for i in *.txt; do diff -ub $i ../old/$i; done > ../diff.txt 

Edit: Fix diff old/new.