System.Quarter

[NN---]

.NET adapts FP16 as System.Half.
There is a plan for FP8.
My suggestion is to add System.Quarter.

https://www.nextplatform.com/2022/07/07/chip-makers-press-for-standardized-fp8-format-for-ai/

[huoyaoyuan]

Now, the ball is in the IEEE’s court to bring everyone together and create a standard.

Of course we should wait for there to be a standard.

[Joe4evr]

@tannergooding expressed a heavy opinion on this type in the Discord yesterday.

[c-ohle]

@Joe4evr hi, do you have a link to this discussion?
It is interesting for me since I support this type already in my NET 7 System.Numerics extensions solution.
as Float<SizeType8> https://github.com/c-ohle/RationalNumerics

[tannergooding]

The FP8 type discussed in that article is not part of the IEEE 754 floating-point specification and as such a name such as Quarter or even Float8 or Binary8 would not be appropriate as it could theoretically conflict in the future.

That being said, I don't particularly expect that IEEE 754 will ever try to formalize an FP8 type as the use-cases scenarios are incredibly domain and scenario specific.

Given the IEEE 754 format, 1-bit is taken up for the sign and at least 2-bits are taken up for the mantissa (significand). This means that the possible layout for a theoretical binary8 would be:

Layout	Exp Bias	Epsilon at Zero	Epsilon at One	Pi	MaxValue	Notes
S1E5M2	15	0.0000152587890625	0.25	3	57344	bfloat8
S1E4M3	7	0.001953125	0.125	3.25	240	likely ieee 754
S1E3M4	3	0.015625	0.625	3.125	15.5
S1E2M5	1	0.03125	0.3125	3.125	3.9375

Compare to Half where the same is:

Layout	Exp Bias	Epsilon at Zero	Epsilon at One	Pi	MaxValue
S1E5M10	15	0.000000059604644775390625	0.0009765625	3.140625	65504

NOTE: These values can be computed manually using the algorithm: -1^s * 2^(exponent - bias) * (implicit + 2^(1-mantissaBitCount) * mantissa)

When the exponent is 0, implicit is 0 and mantissaBitCount is exactly the number specified by M#. Otherwise implicit is 1 and the mantissaBitCount is one higher.

---

What this basically comes down to is that Double provides ~15.65 digits of precision, Single provides ~6.9 digits of precision, and Half provides ~3 digits of precision.

A theoretical float8 however provides ~0.47, 0.84, 1.17, or 1.49 digits of precision. The two formats that are actually semi-usable in terms of range and similarity to other existing IEEE 754 or bfloat formats both provide less than 1 digit of average accuracy making them very poor choices for actual usage.

These types "work" for the ML scenarios because the models they are used in are effectively designed to work around the lack of precision. Such a type would really only be suited for use in those domain specific scenarios and should not be part of the BCL.

The fact that PI for bfloat8 becomes 3 and 3.25 for the most likely of the 4 formats that could be the IEEE 754 compatible one if they ever introduced it really helps emphasize and drive home how much inaccuracy is able to creep in. Let alone that representing the value of 100 itself (where it is 96 for E5M2/E4M3 and Infinity for E3M4/E2M5) becomes increasingly difficult and has other implications in very basic usage of the type.