Add alpha blending

[LeaVerou]

We should have a method for alpha blending. This is required to be able to do contrast calculations properly (see w3c/csswg-drafts#7358 ).

Things to decide:

• Naming: Perhaps overlay() or over()? Or overlayOn() or just on() which is more clear as a Color method, but not so much in the procedural API (See overlayOn(color1, color2) vs color1.overlayOn(color2)). Whatever we come up with should make sense in both APIs, so I’m leaning toward over() which is also the operator name.
• Is there any reason for it to take more than 2 arguments? My understanding is that the operator is associative.

The math seems pretty straightforward. @svgeesus are there any considerations when applying this to other color spaces? What color space do we use for the result? I guess the color space of the color being overlaid?

[svgeesus]

Yes, we need to have blending and compositing available.

The W3C Compositing spec is old, RGB-only, and does compositing in a broken way which happens to be compatible with what browsers do and Adobe Photoshop does, or used to do.

It works in sRGB (or technically any RGB space) but luminance uses the old NTSC primaries (yes, really)

Lum(C) = 0.3 x Cred + 0.59 x Cgreen + 0.11 x Cblue

That means we need to calculate a separate "CSS blend luminance" which is not Y in CIE XYZ /facepalm

https://drafts.fxtf.org/compositing-1/

Be careful when choosing names, because we want to be able to add the blend-modes and source-over is the usual Porter-Duff name for what CSS blending calls normal which is the initial value of mix-blend-mode.

Is there any reason for it to take more than 2 arguments? My understanding is that the operator is associative.

Like mixing, compositing is defined on two arguments and compositing more than that requires specifying and order of operations for several two-argument steps.

In addition to all that broken compositing in gamma-encoded sRGB space we would also want to make available compositing in linear-light xyz-d65 which is what is used for commercial 2D and 3D graphics.

[LeaVerou]

Let's focus on alpha blending for now please. Sure, we shouldn't make any decisions that make it harder to implement the other operators and blending modes later on, but the immediate need is for alpha blending, that is far more common than any other operators, and we don't need to figure out the API for blending modes to add alpha blending.

The W3C compositing spec is old, sure, but surely this is a solved problem in the literature? So we convert to XYZ and do it there, then convert back to the color space of the first color?

[svgeesus]

surely this is a solved problem in the literature? So we convert to XYZ and do it there, then convert back to the color space of the first color?

Yes, that is the correct way.

It doesn't predict the actual contrast you will get in a browser, of course.

[svgeesus]

the immediate need is for alpha blending, that is far more common than any other operators

All of the blend modes are doing alpha blending.

[svgeesus]

Sure, we shouldn't make any decisions that make it harder to implement the other operators and blending modes later on

Exactly. So the correct approach is to implement the General Formula for Compositing and Blending:

Apply the blend in place

Cs = (1 - αb) x Cs + αb x B(Cb, Cs)

Composite

Co = αs x Fa x Cs + αb x Fb x Cb

and then provide useful defaults.

For the blend operator B, the default is normal which is simply

B(Cb, Cs) = Cs

and for source-over compositing,

Fa =1
Fb = 1 – αs

[facelessuser]

Let's focus on alpha blending for now please.

As someone that has already gone down this road, if you add support for source-over compositing and normal alpha blending, you have everything you need to do the others, you can just swap out different compositing and blend methods. So the key is just to make sure the compositing step and the blend step is generic. I think that is the main point that is being conveyed.

Though I had some missteps in the beginning 😅, I can pretty much mimic what browsers are doing now.

[LeaVerou]

@svgeesus But in which color space? Will XYZ work well here? How does this change once we support CMYK spaces?

@facelessuser I see your point, what I meant was, from an API design pov, alpha compositing should be quick and easy to specify, not just a parameter of a more general compositing API.

Though I had some missteps in the beginning 😅, I can pretty much mimic what browsers are doing now.

What missteps did you have?

[svgeesus]

But in which color space? Will XYZ work well here?

For browser-compatible simple alpha source-over blending, premultiplied sRGB will work (oog values may need to be clipped, needs investigation of how the equations react to extended values)

For browser-compatible other blend modes, sRGB plus a bogus NTSC-luma will work; the luma is only needed for the non-seperable blend modes:

• hue
• saturation
• color
• luminosity (sic)

For higher quality physical light normal blending and source-over compositing, premultiplied XYZ-D65 will be fine

For compositing in CMYK then a) are you crazy and b) read the PDF spec for the full horror and c) have a drink with @faceless2 to get extended horror. Also to actually do compositing that is fully general and includes some CMYK then we would need ICC support to be able to get the Lab values. So, not for now.

[svgeesus]

From an API design pov, alpha compositing should be quick and easy to specify, not just a parameter of a more general compositing API.

An API design would have two methods, blend and composite. Each would take the source and destination colors as parameters plus an optional options object where one could specify the blend mode and the compositing operator and the blend colorspace, and these would default to normal and source-over and either xyz-d65 or srgb respectively.

[LeaVerou]

For browser-compatible simple alpha source-over blending, premultiplied sRGB will work (oog values may need to be clipped, needs investigation of how the equations react to extended values)

For browser-compatible other blend modes, sRGB plus a bogus NTSC-luma will work.

For higher quality physical light normal blending and source-over compositing, premultiplied XYZ-D65 will be fine

Let's use XYZ-D65 then, and allow for a parameter if people want to produce shittier results that are browser-compatible (or not, what's the use case?).

For compositing in CMYK then a) are you crazy and b) read the PDF spec for the full horror and c) have a drink with @faceless2 to get extended horror. Also to actually do compositing that is fully general and includes some CMYK then we would need ICC support to be able to get the Lab values. So, not for now.

Of course we'd need ICC support to do CMYK properly, but that's in scope for the future, and wanting to overlay a CMYK value with transparency over another is not a crazy ask. Once we do have Lab values, does alpha blending work the same way?

[svgeesus]

Once we do have Lab values, does alpha blending work the same way?

For the proper way: yes. Lab-D50 → XYZ-D50 → XYZ-D65 and then as normal

For the web-compatible way, Lab-D50 → XYZ-D50 → XYZ-D65 → srgb-linear → sRGB and hope

[facelessuser]

What missteps did you have?

• I forgot to undo the premultiplication after the blending was done. This broke the associative nature. That was just a dumb mistake on my part.
• It also seems that some of the operator algorithms can exceed 0 and 1 for αo (Lighter for instance). I initially didn't clamp them, and instead applied them, and then the final color would have the alpha clamped. But I realized that to mimic browsers, I needed to clamp αo then apply it.

EDIT: Just to clarify, clamping αo is specifically related to premultiplication as well, and in my case, I was using it to undo premultiplication on each channel without first clamping it to a realistic 0 - 1 range. Basically, all my issues were related to undoing premultiplication.