Branch target alignment on Apple Silicon

[filipnavara]

PR #59828 started enforcing 32-byte alignment for methods with loops on ARM64 based on the Neoverse N1 optimization guide:

runtime/src/coreclr/jit/emit.cpp

Lines 6775 to 6785 in 16fe4d4

	#if defined(TARGET_XARCH) || defined(TARGET_ARM64)
	// For x64/x86/arm64, align methods that are "optimizations enabled" to 32 byte boundaries if
	// they are larger than 16 bytes and contain a loop.
	//
	if (emitComp->opts.OptimizationEnabled() &&
	(!emitComp->opts.jitFlags->IsSet(JitFlags::JIT_FLAG_PREJIT) || comp->IsTargetAbi(CORINFO_NATIVEAOT_ABI)) &&
	(emitTotalHotCodeSize > 16) && emitComp->fgHasLoops)
	{
	codeAlignment = 32;
	}
	#endif

This goes contrary to the Apple Silicon CPU Optimization Guide, section 4.4.3 Branch Target Alignment. Apple specifically states that software alignment of branch targets is unnecessary and sometimes detrimental due to the alignment capabilities of the processor. The guidance is to not align branch targets and favor smaller code size.

Aside for performance implications this alignment makes the size of NativeAOT code on iOS/macOS bigger due to unnecessary alignment.

[dotnet-policy-service]

Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[ivanpovazan]

Regarding:

Aside for performance implications this alignment makes the size of NativeAOT code on iOS/macOS bigger due to unnecessary alignment.

Do you by any chance have a measure of perf/size implications caused by this?

[filipnavara]

Do you by any chance have a measure of perf/size implications caused by this?

Very rough ballpark estimate is 50 Kb for empty MAUI app. I'll actually need to implement the change to measure something more specific. I noticed it when measuring unrelated change where 8-byte changes in prolog/epilog size resulted in 32-byte changes in object file.

Not sure how to easily benchmark it without running the whole dotnet/performance test suite and looking for improvements/regressions. Do we have some automated runs on Apple Silicon or does it have to be done manually?

[filipnavara]

Tested on dotnet new maui targeting ios-arm64. Removing the method alignment results in -24.6 Kb. Removing the actual loop alignment inside JIT gets additional 2.4 Kb but it triggers at least three different asserts in JIT.

Note that the default ILC method alignment is 16 bytes, so the impact would be bigger for <OptimizationPreference>Size</OptimizationPreference> where the default method alignment is 4 bytes.

[filipnavara]

With --Os the size difference is -45.2 Kb with disabling the loop alignment and related code alignment.

[ivanpovazan]

Thanks a lot for checking it out! We should also run some perf benchmarks to see how much it impacts the speed.
Regarding:

Not sure how to easily benchmark it without running the whole dotnet/performance test suite and looking for improvements/regressions. Do we have some automated runs on Apple Silicon or does it have to be done manually?

/cc: @matouskozak

[matouskozak]

Thanks a lot for checking it out! We should also run some perf benchmarks to see how much it impacts the speed. Regarding:

Not sure how to easily benchmark it without running the whole dotnet/performance test suite and looking for improvements/regressions. Do we have some automated runs on Apple Silicon or does it have to be done manually?

/cc: @matouskozak

I'm not aware of any automated runs on macOS apple silicon, only Windows and Linux x64/arm64 machines. If you have any specific scenarios that you expect to be affected, you can use the benchmarking script https://github.com/dotnet/performance/blob/main/scripts/BENCHMARKS_LOCAL_README.md which allows you to run specific microbenchmarks from the dotnet/performance repo locally.

@EgorBo do you know if CoreCLR perf is tested on Apple Silicon?

[EgorBo]

@EgorBo do you know if CoreCLR perf is tested on Apple Silicon?

Sadly, we don't test on macOS, so before making changes like this we should include macOS into perf testing first

[tannergooding]

It's also worth noting that this is going to end up being a microarchitecture specific optimization. Most of the Arm optimization manuals (such as for Neoverse V1 or Cortex-A75) explicitly call out that branch instruction and branch target instruction alignment/density impacts performance. They typically recommend not placing more than four branch instructions within an aligned 32-byte window, etc.

So doing this may increase the testing complexity matrix and may change in the future. Notably, there's also complexity in that some architectures differentiate loop target alignment from branch target alignment and have different recommendations between the two of them.

---

If we did something here, it might be beneficial to ensure we get the right flexibility added so that things can be customized as appropriate.

Some microarchitectures benefit the most from no alignment, some from 16, some from 32, and even some of the latest from 64-byte alignment instead. For example, both the latest AMD and Intel optimization guides recommend using a 64-byte code window consideration, while somewhat older ones recommend 32-bytes instead (which is what the JIT currently does, but only considering loop starts), and "legacy cpus" recommend using 16-bytes. -- The root consideration here is not necessarily to align the loop, but rather to fit as much of the loop body into a single "instruction fetch and/or decode window" as possible. Hence the official guidance details considering aligning either the loop start of a hot loop to the beginning of the aligned boundary or the loop end to the end of an aligned boundary, with the latter often being preferred if you have to pick aligning one vs the other (something we don't even try to consider today). If a loop already fits entirely in the optimal window, then there can be much lesser benefit to doing any alignment and code size can be saved.