Why does a stackalloc statement before a foreach loop prevent the loop from being optimized?

[KPHIBYE]

I ran benchmarks to test the devirtualization and static readonly constant folding of .NET 10 that were described in the Performance Improvements in .NET 10 Dev Blog by Stephen Toub but observed that the two don't seem to work as I would have expected in the following scenario:

using System;
using System.Linq;
using System.Collections.Generic;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace Benchmarks;

[MemoryDiagnoser(displayGenColumns: false)]
[HideColumns("Job", "RatioSD", "Alloc Ratio")]
[SimpleJob(RuntimeMoniker.Net90)]
[SimpleJob(RuntimeMoniker.Net10_0)]
public class StaticReadonlyStackallocBenchmarks
{
    private const bool ConstUseStackalloc = false;
    private static readonly bool UseStackalloc = ConstUseStackalloc;
    private static readonly List<int> Ints = Enumerable.Range(1, 1_000).ToList();

    [Benchmark(Baseline = true)]
    public int Baseline()
    {
        int sum = 0;
        foreach (int i in (IEnumerable<int>)Ints) sum += i;
        return sum;
    }

    [Benchmark]
    public int Const()
    {
        Span<byte> buffer = ConstUseStackalloc ? stackalloc byte[1] : stackalloc byte[0];

        int sum = 0;
        foreach (int i in (IEnumerable<int>)Ints) sum += i;
        return sum;
    }

    [Benchmark]
    public int StaticReadonly()
    {
        Span<byte> buffer = UseStackalloc ? stackalloc byte[1] : stackalloc byte[0];

        int sum = 0;
        foreach (int i in (IEnumerable<int>)Ints) sum += i;
        return sum;
    }

    [Benchmark]
    public int StaticReadonlySumMethod()
    {
        Span<byte> buffer = UseStackalloc ? stackalloc byte[1] : stackalloc byte[0];

        return Sum(Ints);

        static int Sum(IEnumerable<int> ints)
        {
            int sum = 0;
            foreach (int i in ints) sum += i;
            return sum;
        }
    }
}

BenchmarkDotNet v0.15.6, Windows 10 (10.0.19045.6456/22H2/2022Update)
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET SDK 10.0.100
  [Host]    : .NET 9.0.11 (9.0.11, 9.0.1125.51716), X64 RyuJIT x86-64-v3
  .NET 10.0 : .NET 10.0.0 (10.0.0, 10.0.25.52411), X64 RyuJIT x86-64-v3
  .NET 9.0  : .NET 9.0.11 (9.0.11, 9.0.1125.51716), X64 RyuJIT x86-64-v3


| Method                  | Runtime   | Mean       | Error     | StdDev    | Ratio | Allocated |
|------------------------ |---------- |-----------:|----------:|----------:|------:|----------:|
| Baseline                | .NET 10.0 |   592.5 ns |   9.91 ns |  11.79 ns |  1.00 |         - |
| Const                   | .NET 10.0 |   574.2 ns |   6.81 ns |   5.32 ns |  0.97 |         - |
| StaticReadonly          | .NET 10.0 | 8,016.5 ns | 118.79 ns | 111.11 ns | 13.54 |      40 B |
| StaticReadonlySumMethod | .NET 10.0 |   576.4 ns |   9.86 ns |   9.22 ns |  0.97 |         - |
|                         |           |            |           |           |       |           |
| Baseline                | .NET 9.0  | 1,932.3 ns |  21.68 ns |  18.10 ns |  1.00 |      40 B |
| Const                   | .NET 9.0  | 1,931.7 ns |  17.87 ns |  16.71 ns |  1.00 |      40 B |
| StaticReadonly          | .NET 9.0  | 7,720.0 ns |  81.43 ns |  72.19 ns |  4.00 |      40 B |
| StaticReadonlySumMethod | .NET 9.0  | 1,950.8 ns |  27.44 ns |  21.42 ns |  1.01 |      40 B |

I understand that the compiler removes the whole first line of the Const method via dead-code elimination and that this method is thus equal to Baseline but this is sadly where my expertise ends. I hope that someone can answer me as many of the following questions as possible:

1. What prevents the JIT from applying optimizations in StaticReadonly? The issue cannot be explained by missing devirtualization alone, since the same behavior appears in the .NET 9 runs where devirtualization does not occur.
2. Why doesn't the JIT treat UseStackalloc as constant and eliminate the first line of the StaticReadonly method, as the compiler does for the Const method?
3. Why does moving the foreach loop to a separate method enable performance optimizations, including devirtualization?

I am thankful for all answers and everything new that I am able to learn 🙏

[EgorBo]

Thanks for reporting this issue, it seems to be running into an edge case - stackalloc inside a method with loops, jit conservatively immediately self-promotes itself to the final tier (FullOpts):

**** JIT Tier0 jit request switching to Tier1 because: OSR can't handle localloc

It is something we want to eventually fix, for now, we can workaround it - JIT shouldn't give up on small constant sized stackallocs (<=32b) as we promote those to locals.

[KPHIBYE]

Thank you for your reply. It helped me understand the problem better. But what about cases where the stackalloc would be larger than 32b and why doesn't the JIT treat static readonly as constant and eliminate the stackalloc altogether in my examples? The provided benchmarks are only a minimal reproduction of the underlaying problem. My actual use case is similar to the following code and would not benefit from your proposed optimization. Do I really have to extract the loops into separate methods to work around this issue and help the JIT out?

public sealed class MyClass<T>
{
    private static readonly int StackallocThreshold = 512;
    private static readonly int RequiredBytes = 1024; // this value depends on T and is calculated at runtime
    
    private readonly byte[] _bytes = new byte[RequiredBytes];
    
    public void DoStuff(IEnumerable<int> data)
    {
        Span<byte> buffer = RequiredBytes <= StackallocThreshold ? stackalloc byte[RequiredBytes] : new byte[RequiredBytes];

        foreach (int i in data)
        {
            // populate buffer
        }
        
        // do stuff with buffer and _bytes
    }
}

[EgorBo]

why doesn't the JIT treat static readonly as constant and eliminate the stackalloc altogether in my examples?

@KPHIBYE in ConstUseStackalloc case, Roslyn eliminates your localloc at C# compilation time, so JIT never even sees it and never panics by doing that transition (so it cannot use PGO and devirtualize/deabstract that loop).

So like I said: stackallocs are problematic in methods with loops, run this benchmark:

[Benchmark]
public int Const()
{
    Span<byte> buffer = ConstUseStackalloc ? stackalloc byte[1] : stackalloc byte[0];

    int sum = 0;
    foreach (int i in (IEnumerable<int>)Ints) sum += i;
    return sum;
}

[Benchmark]
public int StaticReadonly()
{
    Span<byte> buffer = UseStackalloc ? stackalloc byte[1] : stackalloc byte[0];

    int sum = 0;
    foreach (int i in (IEnumerable<int>)Ints) sum += i;
    return sum;
}

[Benchmark]
public int StaticReadonly_hack()
{
    Span<byte> buffer = UseStackalloc ? stackalloc byte[1] : stackalloc byte[0];

    return Hack();
    [MethodImpl(MethodImplOptions.NoInlining)]
    static int Hack()
    {
        int sum = 0;
        foreach (int i in (IEnumerable<int>)Ints) sum += i;
        return sum;
    }
}

We'll fix this issue eventually, just can't promise it will happen next version.

[KPHIBYE]

Thank you for reminding me to add [MethodImpl(MethodImplOptions.NoInlining)] to the loop-method. I would have forgotten it otherwise. I am also relieved that the .NET team is aware of this problem and hope that it can also find a solution for stackallocs >32b. And of course, I am aware that these optimizations take time to be developed and tested. Is there an existing issue that I can follow to keep track of the progress?

I now also believe to understand why static readonly fields are not treated as constants by the JIT inside methods that contain a stackalloc and a loop. The documentation of MethodImplOptions.AggressiveOptimization mentions that going straight to Tier1, which you stated is happening here, prevents optimizations based on initialized classes which static readonly fields should be a part of.

Methods that apply this attribute bypass the first tier of tiered compilation and therefore don't benefit from optimizations that rely on tiered compilation. Those optimizations include dynamic PGO and optimizations based on initialized classes.

Again, thank you for your explanation. It really helped me to better understand why my code is not behaving as expected.