[API Proposal]: Blackhole

[timcassell]

Background and motivation

A JIT intrinsic API that does nothing. Similar to Java's Blackhole. Blackhole is used to prevent dead code elimination and out-of-order instructions generated by a compiler (not by the CPU, so it doesn't emit any barriers). The actual call does nothing, and is stripped out of the assembly.

The purpose of this API is to make it easier/possible to benchmark sensitive operations. Without this API, heavier alternatives are required like calling a method with NoInlining, or writing to a field (see Consumer class in BDN), which can spoil the results of the measurement. The cost of this has already been observed (#89940) (also credit to @EgorBo for this idea in that issue).

API Proposal

namespace System.Runtime.CompilerServices;

public static class Blackhole
{
    public static void Consume<T>(scoped T value) where T : allows ref struct;
    public static void Consume<T>(scoped ref readonly T value) where T : allows ref struct;
    public static unsafe void Consume(void* value);
}

API Usage

Old code:

public class HalfCast
{
    private readonly Consumer consumer = new();
    
    // 4 operations per invoke averages the cost of the 4 operations
    [Benchmark(OperationsPerInvoke = 4)]
    [Arguments(6.097555E-05f, 12345.0f, 65520.0f, float.NaN)]
    public void SingleToHalfAverage(float arg1, float arg2, float arg3, float arg4)
    {
        // Oh no, the consumer's volatile write interferes with the results!
        consumer.Consume((Half) arg1);
        consumer.Consume((Half) arg2);
        consumer.Consume((Half) arg3);
        consumer.Consume((Half) arg4);
    }
}

New code:

public class HalfCast
{
    // 4 operations per invoke averages the cost of the 4 operations
    [Benchmark(OperationsPerInvoke = 4)]
    [Arguments(6.097555E-05f, 12345.0f, 65520.0f, float.NaN)]
    public void SingleToHalfAverage(float arg1, float arg2, float arg3, float arg4)
    {
        // Yay, we get accurate results!
        Blackhole.Consume((Half) arg1);
        Blackhole.Consume((Half) arg2);
        Blackhole.Consume((Half) arg3);
        Blackhole.Consume((Half) arg4);
    }
}

Alternative Designs

Without any new APIs - Pass the value to a method with NoInlining, or write to a field. Both of these have some overhead.

Alternative new APIs (discussed in comments below):

namespace System.Runtime.CompilerServices;

public static class BlackBox
{
    public static T Identity<T>(scoped T value) where T : allows ref struct;
    public static ref readonly T Identity<T>(scoped ref readonly T value) where T : allows ref struct;
    public static unsafe void* Identity(void* value);
}

Risks

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

[hamarb123]

I don't think there'd be any point to a T* version if we have a void* version (since the conversion is a no-op at IL level). It would also be good to have a version that takes ref T though (still with the where T : allows ref struct) so we can consume references without having to create a ByRef struct just to pass to this API or similar.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime-compilerservices
See info in area-owners.md if you want to be subscribed.

[timcassell]

I don't think there'd be any point to a T* version if we have a void* version (since the conversion does not need to be performed at IL level).

This is true, but the T* version is just more ergonomic. Otherwise you'd have to add the cast to your code. [Edit] I just tried it, and I was wrong. No cast is necessary. I will update the proposal to remove that.

Ideally it would just be a single API with an extra allows pointer anti-constraint, but that doesn't exist currently.

It would also be good to have a version that takes ref T though (still with the where T : allows ref struct) so we can consume references without having to create a ByRef struct just to pass to this API or similar.

That's not necessary.

ref int target = ref myArray[0];
Blackhole.Consume(target);

[hamarb123]

This is true, but the T* version is just more ergonomic. Otherwise you'd have to add the cast to your code. Ideally it would just be a single API with an extra allows pointer anti-constraint, but that doesn't exist currently.

That's not true - you don't have to explicitly cast to void* (it's an implicit cast in c#).

That's not necessary.

ref int target = ref myArray[0];
Blackhole.Consume(target);

This is not behaviourally or conceptually the same as Blackhole.Consume(ref target) - the former explicitly dereferences the byref (which is not a no-op necessarily, for example it can throw in some cases), and is equivalent to (ignoring array covariance) Blackhole.Consume(myArray[0]) from the runtime's perspective. Blackhole.Consume(ref x) would consume the by-reference itself, whereas Blackhole.Consume(x) just consumes x.

[timcassell]

You are correct, I updated the proposal.

[timcassell]

Should it be in T rather than ref T? ref can go to in, but not vice-versa without Unsafe.

[hamarb123]

From the runtime's perspective, in T and ref T are the same. This may lead to the issue where you pass something by in and expect the method to treat it as if the ref can't be mutated, but the runtime will most likely not implement it this way (as they're all just T& to it). I would add it to the open questions section, and leave it with ref (since it's technically more correct).

Also, ref readonly would be better suited for this than in.

[jkotas]

prevent dead code elimination and out-of-order instructions generated by a compiler (not by the CPU

Is this actually useful if it does not prevent out-of-order execution done by the CPU?

I think writing the value into a globally visible field to establish data dependency has better fidelity for micro-benchmarking purposes.

[timcassell]

Is this actually useful if it does not prevent out-of-order execution done by the CPU?

I think writing the value into a globally visible field to establish data dependency has better fidelity for micro-benchmarking purposes.

The idea is for this to be zero cost. As I understand it, barriers that prevent cpu instruction reordering are not free. Anyway, I don't think it's unrealistic for real-world code to do calculations without barriers (storing results in locals instead of fields), so the micro-benchmarks should be fine, I think.

Also, refs and ref structs cannot be written to globally visible fields, so even if we were to have that behavior, the runtime would need to treat them the same. That is currently not possible without runtime support.