Skip to content

Add BFloat16 #98643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 77 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
589afe0
Add api for BFloat16
huoyaoyuan Feb 18, 2024
312d051
Creating
huoyaoyuan Feb 18, 2024
5e1c981
Equals and GetHashCode
huoyaoyuan Feb 18, 2024
1fb4765
Comparison
huoyaoyuan Feb 18, 2024
fc05d3b
Constants and comment
huoyaoyuan Feb 18, 2024
152fe99
Xml doc
huoyaoyuan Feb 18, 2024
25a16e7
Using rounding for cast
huoyaoyuan Feb 18, 2024
50d90aa
Ref source
huoyaoyuan Feb 18, 2024
559f2e0
Simple tests
huoyaoyuan Feb 18, 2024
b24839c
Conversion tests
huoyaoyuan Feb 19, 2024
8284526
Stripping sign is redundant
huoyaoyuan Feb 19, 2024
8e32e71
Fix test copied from Half
huoyaoyuan Feb 19, 2024
4bd266e
Fix conversion test cases
huoyaoyuan Feb 19, 2024
6df00e6
Constants and well-known values
huoyaoyuan Feb 21, 2024
ff295fd
Categorizing methods
huoyaoyuan Feb 21, 2024
09af2b2
Reorder conversion members
huoyaoyuan Feb 21, 2024
1a8f0ad
Operators batch 1
huoyaoyuan Feb 21, 2024
c9fc867
Operators batch 2
huoyaoyuan Feb 21, 2024
e9fc0f8
TryConvert
huoyaoyuan Feb 21, 2024
c967aa5
Operators batch 3
huoyaoyuan Feb 21, 2024
17c13c0
Parsing and formatting
huoyaoyuan Feb 21, 2024
ad780a0
Add comments about how to determine parse and format info
huoyaoyuan Feb 21, 2024
c01949f
Add missing interface implementations
huoyaoyuan Feb 21, 2024
b63c1df
NumberBufferLength
huoyaoyuan Feb 21, 2024
754a3c8
Add more comment
huoyaoyuan Feb 22, 2024
bcc260f
Correct MinFastFloatDecimalExponent
huoyaoyuan Feb 24, 2024
5a3d200
Add explicit conversion to
huoyaoyuan Feb 24, 2024
c420dd3
Explicit convert from
huoyaoyuan Feb 24, 2024
13e65d1
Fullfill casting operators
huoyaoyuan Feb 24, 2024
8c5f546
Fullfill some formatting
huoyaoyuan Feb 24, 2024
0cb3932
Apply suggestions from code review
huoyaoyuan Apr 6, 2024
b615e68
Merge branch 'main' into BFloat16
huoyaoyuan Apr 6, 2024
8f70d91
Generic DiyFp
huoyaoyuan Feb 24, 2024
2458dd8
Generic Grisu3
huoyaoyuan Feb 24, 2024
a8bb94b
Generic Dragon4
huoyaoyuan Feb 24, 2024
9644914
Add MaxRoundTripDigits to MaxPrecisionCustomFormat to FormatInfo
huoyaoyuan Feb 25, 2024
a29db5c
Generic FormatFloat
huoyaoyuan Feb 25, 2024
a8a8a49
Adapt with existing FP types
huoyaoyuan Feb 25, 2024
2fd392f
Merge branch 'fp-formatting-generic' into BFloat16
huoyaoyuan Apr 6, 2024
f1582e7
Adapt formatting traits
huoyaoyuan Apr 6, 2024
eace3a6
Use generic format and delete Number.BFloat16
huoyaoyuan Apr 6, 2024
abd1e80
Update ref source
huoyaoyuan Apr 6, 2024
62156c9
Merge branch 'main'
huoyaoyuan May 3, 2024
e8012c9
Enable constant value tests
huoyaoyuan May 3, 2024
f711f8d
IsFinite/IsNaN
huoyaoyuan May 3, 2024
08168ff
IsPositive/IsNegative/IsSubnormal
huoyaoyuan May 3, 2024
d59a8c5
ToDouble
huoyaoyuan May 3, 2024
eb6dc47
Merge branch 'main' into BFloat16
huoyaoyuan May 23, 2024
25b7684
Merge branch 'main'
huoyaoyuan Jun 4, 2024
832651e
Merge branch 'main' into BFloat16
huoyaoyuan Jun 9, 2024
a07fe96
Fix test case
huoyaoyuan Jun 9, 2024
f9c35d3
Add double conversion test
huoyaoyuan Jun 9, 2024
4059b66
Parse tests
huoyaoyuan Jun 9, 2024
4b4d1a5
Formatting tests
huoyaoyuan Jun 9, 2024
6ed52f5
RoundTripping tests
huoyaoyuan Jun 9, 2024
14b0d85
Port float->Half conversion algorithm to double->BFloat16 to handle U…
huoyaoyuan Jun 10, 2024
ea1dd5f
Port function tests from Half
huoyaoyuan Jun 10, 2024
dfd49c8
Convert the precesion of test cases.
huoyaoyuan Jun 10, 2024
f5461ac
Merge branch 'main' into BFloat16
danmoseley Dec 2, 2024
daaec69
Merge branch 'main' into BFloat16
huoyaoyuan Dec 29, 2024
9938e8b
Align with TryWriteBig/LittleEndian
huoyaoyuan Dec 29, 2024
b889417
Remove redundant 'partial'
huoyaoyuan Dec 31, 2024
1282c85
Merge branch 'main' into BFloat16
tannergooding Jan 27, 2025
e6dd118
Merge branch 'main' into BFloat16
huoyaoyuan Feb 6, 2025
922f411
Use DefaultParseStyle
huoyaoyuan Feb 6, 2025
86dce2d
Fill conversion in signed integer
huoyaoyuan Feb 6, 2025
9f729a3
Fill conversion in unsigned integer and floating point
huoyaoyuan Feb 6, 2025
6baf940
Add conversion for S.R.Numerics
huoyaoyuan Feb 6, 2025
fb88f8e
Use float member function instead of MathF
huoyaoyuan Feb 6, 2025
d697344
Fill conversion in decimal
huoyaoyuan Feb 7, 2025
4e83ad9
Add conversion for NFloat
huoyaoyuan Feb 7, 2025
d58c80a
Use soft rounding for uint->bf16
huoyaoyuan Feb 9, 2025
8639fa3
Generic math rounding from unsigned and signed integer
huoyaoyuan Feb 10, 2025
34b1d07
Cleanup helper methods
huoyaoyuan Feb 10, 2025
8adbeb2
Add integer rounding tests
huoyaoyuan Feb 10, 2025
1f2653f
Move helpers and fix comment
huoyaoyuan Feb 11, 2025
9046622
Update comment
huoyaoyuan Feb 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3346,7 +3346,7 @@
</data>
<data name="RFLCT_Targ_ITargMismatch_WithType" xml:space="preserve">
<value>Object type {0} does not match target type {1}.</value>
</data>
</data>
<data name="RFLCT_Targ_StatFldReqTarg" xml:space="preserve">
<value>Non-static field requires a target.</value>
</data>
Expand Down Expand Up @@ -4310,4 +4310,7 @@
<data name="WasmThreads_BlockingWaitNotSupportedOnJSInterop" xml:space="preserve">
<value>Blocking wait is not supported on the JS interop threads.</value>
</data>
</root>
<data name="Arg_MustBeBFloat16" xml:space="preserve">
<value>Object must be of type BFloat16.</value>
</data>
</root>
Original file line number Diff line number Diff line change
Expand Up @@ -587,6 +587,7 @@
<Compile Include="$(MSBuildThisFileDirectory)System\Number.Grisu3.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Number.NumberToFloatingPointBits.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Number.Parsing.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Numerics\BFloat16.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Numerics\BitOperations.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Numerics\Matrix3x2.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Numerics\Matrix3x2.Impl.cs" />
Expand Down
148 changes: 148 additions & 0 deletions src/libraries/System.Private.CoreLib/src/System/Numerics/BFloat16.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

namespace System.Numerics
{
/// <summary>
/// Represents a shortened (16-bit) version of 32 bit floating-point value (<see cref="float"/>).
/// </summary>
public readonly struct BFloat16
: IComparable,
IComparable<BFloat16>,
IEquatable<BFloat16>
{
private const ushort EpsilonBits = 0x0001;

private const ushort MinValueBits = 0xFF7F;
private const ushort MaxValueBits = 0x7F7F;

/// <summary>
/// Represents the smallest positive <see cref="BFloat16"/> value that is greater than zero.
/// </summary>
public static BFloat16 Epsilon => new BFloat16(EpsilonBits);

/// <summary>
/// Represents the smallest possible value of <see cref="BFloat16"/>.
/// </summary>
public static BFloat16 MinValue => new BFloat16(MinValueBits);

/// <summary>
/// Represents the largest possible value of <see cref="BFloat16"/>.
/// </summary>
public static BFloat16 MaxValue => new BFloat16(MaxValueBits);

internal readonly ushort _value;

internal BFloat16(ushort value) => _value = value;

// Casting

/// <summary>Explicitly converts a <see cref="float" /> value to its nearest representable <see cref="BFloat16"/> value.</summary>
/// <param name="value">The value to convert.</param>
/// <returns><paramref name="value" /> converted to its nearest representable <see cref="BFloat16"/> value.</returns>
public static explicit operator BFloat16(float value)
{
uint bits = BitConverter.SingleToUInt32Bits(value);
uint upper = bits >> 16;
// Only do rounding for finite numbers
if (float.IsFinite(value))
{
uint lower = bits & 0xFFFF;
uint sign = upper & 0x8000;
// Strip sign from upper
upper &= 0x7FFF;
// Determine the increment for rounding
// When upper is even, midpoint (0x8000) will tie to no increment, which is effectively a decrement of lower
uint lowerShift = (~upper) & (lower >> 15) & 1; // Upper is even & lower>=0x8000 (not 0)
lower -= lowerShift;
uint increment = lower >> 15;
// Do the increment, MaxValue will be correctly increased to Infinity
upper += increment;
// Put back sign with upper bits and done
upper |= sign;
}
return new BFloat16((ushort)upper);
}

/// <summary>Explicitly converts a <see cref="double" /> value to its nearest representable <see cref="BFloat16"/> value.</summary>
/// <param name="value">The value to convert.</param>
/// <returns><paramref name="value" /> converted to its nearest representable <see cref="BFloat16"/> value.</returns>
public static explicit operator BFloat16(double value) => (BFloat16)(float)value;

/// <summary>Explicitly converts a <see cref="BFloat16" /> value to its nearest representable <see cref="float"/> value.</summary>
/// <param name="value">The value to convert.</param>
/// <returns><paramref name="value" /> converted to its nearest representable <see cref="float"/> value.</returns>

public static explicit operator float(BFloat16 value) => BitConverter.Int32BitsToSingle(value._value << 16);

/// <summary>Explicitly converts a <see cref="BFloat16" /> value to its nearest representable <see cref="double"/> value.</summary>
/// <param name="value">The value to convert.</param>
/// <returns><paramref name="value" /> converted to its nearest representable <see cref="double"/> value.</returns>
public static explicit operator double(BFloat16 value) => (double)(float)value;

// BFloat is effectively a truncation of Single, with lower 16 bits of mantissa truncated.
// Delegating all operations to Single should be correct and effective.

// Comparison

/// <summary>
/// Compares this object to another object, returning an integer that indicates the relationship.
/// </summary>
/// <returns>A value less than zero if this is less than <paramref name="obj"/>, zero if this is equal to <paramref name="obj"/>, or a value greater than zero if this is greater than <paramref name="obj"/>.</returns>
/// <exception cref="ArgumentException">Thrown when <paramref name="obj"/> is not of type <see cref="BFloat16"/>.</exception>
public int CompareTo(object? obj)
{
if (obj is not BFloat16 other)
{
return (obj is null) ? 1 : throw new ArgumentException(SR.Arg_MustBeBFloat16);
}
return CompareTo(other);
}

/// <summary>
/// Compares this object to another object, returning an integer that indicates the relationship.
/// </summary>
/// <returns>A value less than zero if this is less than <paramref name="other"/>, zero if this is equal to <paramref name="other"/>, or a value greater than zero if this is greater than <paramref name="other"/>.</returns>
public int CompareTo(BFloat16 other) => ((float)this).CompareTo((float)other);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is converting to float and doing the comparison "cheaper" than just comparing the raw bits?

Presumably its shl; movd; shl; movd; ucomiss and so is expected cheaper than the if (IsNaN(x) || IsNaN(y)) { return false; } return (x == y) || AreZero(x, y); } and similar patterns for other paths?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it should be, as upcasting is simple shifting without branching. I would do a benchmark later.


/// <inheritdoc cref="IEqualityOperators{TSelf, TOther, TResult}.op_Equality(TSelf, TOther)" />
public static bool operator ==(BFloat16 left, BFloat16 right) => (float)left == (float)right;

/// <inheritdoc cref="IEqualityOperators{TSelf, TOther, TResult}.op_Inequality(TSelf, TOther)" />
public static bool operator !=(BFloat16 left, BFloat16 right) => (float)left != (float)right;

/// <inheritdoc cref="IComparisonOperators{TSelf, TOther, TResult}.op_LessThan(TSelf, TOther)" />
public static bool operator <(BFloat16 left, BFloat16 right) => (float)left < (float)right;

/// <inheritdoc cref="IComparisonOperators{TSelf, TOther, TResult}.op_GreaterThan(TSelf, TOther)" />
public static bool operator >(BFloat16 left, BFloat16 right) => (float)left > (float)right;

/// <inheritdoc cref="IComparisonOperators{TSelf, TOther, TResult}.op_LessThanOrEqual(TSelf, TOther)" />
public static bool operator <=(BFloat16 left, BFloat16 right) => (float)left <= (float)right;

/// <inheritdoc cref="IComparisonOperators{TSelf, TOther, TResult}.op_GreaterThanOrEqual(TSelf, TOther)" />
public static bool operator >=(BFloat16 left, BFloat16 right) => (float)left >= (float)right;

// Equality

/// <summary>
/// Returns a value that indicates whether this instance is equal to a specified <paramref name="other"/> value.
/// </summary>
public bool Equals(BFloat16 other) => ((float)this).Equals((float)other);

/// <summary>
/// Returns a value that indicates whether this instance is equal to a specified <paramref name="obj"/>.
/// </summary>
public override bool Equals(object? obj) => obj is BFloat16 other && Equals(other);

/// <summary>
/// Serves as the default hash function.
/// </summary>
public override int GetHashCode() => ((float)this).GetHashCode();

/// <summary>
/// Returns a string representation of the current value.
/// </summary>
public override string ToString() => ((float)this).ToString();
}
}
23 changes: 23 additions & 0 deletions src/libraries/System.Runtime/ref/System.Runtime.cs
Original file line number Diff line number Diff line change
Expand Up @@ -10590,6 +10590,29 @@ public static void HtmlEncode(string? value, System.IO.TextWriter output) { }
}
namespace System.Numerics
{
public readonly partial struct BFloat16 : System.IComparable, System.IComparable<System.Numerics.BFloat16>, System.IEquatable<System.Numerics.BFloat16>
{
private readonly int _dummyPrimitive;
public static System.Numerics.BFloat16 Epsilon { get { throw null; } }
public static System.Numerics.BFloat16 MaxValue { get { throw null; } }
public static System.Numerics.BFloat16 MinValue { get { throw null; } }
public int CompareTo(System.Numerics.BFloat16 other) { throw null; }
public int CompareTo(object? obj) { throw null; }
public bool Equals(System.Numerics.BFloat16 other) { throw null; }
public override bool Equals(object? obj) { throw null; }
public override int GetHashCode() { throw null; }
public static bool operator ==(System.Numerics.BFloat16 left, System.Numerics.BFloat16 right) { throw null; }
public static explicit operator System.Numerics.BFloat16 (double value) { throw null; }
public static explicit operator double (System.Numerics.BFloat16 value) { throw null; }
public static explicit operator float (System.Numerics.BFloat16 value) { throw null; }
public static explicit operator System.Numerics.BFloat16 (float value) { throw null; }
public static bool operator >(System.Numerics.BFloat16 left, System.Numerics.BFloat16 right) { throw null; }
public static bool operator >=(System.Numerics.BFloat16 left, System.Numerics.BFloat16 right) { throw null; }
public static bool operator !=(System.Numerics.BFloat16 left, System.Numerics.BFloat16 right) { throw null; }
public static bool operator <(System.Numerics.BFloat16 left, System.Numerics.BFloat16 right) { throw null; }
public static bool operator <=(System.Numerics.BFloat16 left, System.Numerics.BFloat16 right) { throw null; }
public override string ToString() { throw null; }
}
public static partial class BitOperations
{
[System.CLSCompliantAttribute(false)]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<Project Sdk="Microsoft.NET.Sdk">
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>$(NetCoreAppCurrent)-windows;$(NetCoreAppCurrent)-unix;$(NetCoreAppCurrent)-browser</TargetFrameworks>
<DefineConstants Condition="'$(TargetOS)' == 'browser'">$(DefineConstants);TARGET_BROWSER</DefineConstants>
Expand Down Expand Up @@ -132,6 +132,7 @@
<Compile Include="System\NullableMetadataTests.cs" />
<Compile Include="System\NullableTests.cs" />
<Compile Include="System\NullReferenceExceptionTests.cs" />
<Compile Include="System\Numerics\BFloat16Tests.cs" />
<Compile Include="System\Numerics\IExponentialFunctionsTests.cs" />
<Compile Include="System\Numerics\IFloatingPointTests.cs" />
<Compile Include="System\Numerics\TotalOrderIeee754ComparerTests.cs" />
Expand Down
Loading