Skip to content

[css-compositing-2] Add plus-lighter to mix-blend-mode and background-blend-mode#444

Merged
cabanier merged 4 commits into
w3c:mainfrom
jakearchibald:plus-lighter-mix-mode
Jan 17, 2022
Merged

[css-compositing-2] Add plus-lighter to mix-blend-mode and background-blend-mode#444
cabanier merged 4 commits into
w3c:mainfrom
jakearchibald:plus-lighter-mix-mode

Conversation

@jakearchibald

@jakearchibald jakearchibald commented Nov 26, 2021

Copy link
Copy Markdown
Contributor

Fixes https://github.com/w3c/csswg-drafts/issues/6821.
Tests web-platform-tests/wpt#31757.

This allows any two elements to be cross-faded together. More details: https://jakearchibald.com/2021/dom-cross-fade/.

I'm not really sure about the difference between "blend" and "composite", particularly since globalCompositeOperation in canvas includes both.

We might eventually get to the stage where mix-blend-mode and globalCompositeOperation have the same set of operations, which is kinda weird since they have different names.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

I'm going to try and write some tests for this, but I haven't written CSS tests before. Wish me luck!

@jakearchibald

Copy link
Copy Markdown
Contributor Author

@tabatkins are the tests in the right format for this? The other tests look like ref tests, but maybe there's a more modern way?

@jakearchibald

Copy link
Copy Markdown
Contributor Author

#446 - wondering if plus-lighter is meaningfully different to lighter. If it turns out 'no', I'll switch to lighter in this PR.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

The more I think about it, moving lighter from a compositing mode to a blend mode is cheating. I'll think of a better way to do this.

@cabanier maybe you have thoughts here? Right now globalCompositingOperation has both blend modes and composite modes. Maybe the right thing to do is make mix-blend-mode the same. However, Chrome is only interested in implementing lighter right now.

Should the spec change be limited to what Chrome's currently prepared to implement, or converge with globalCompositingOperation hoping that browsers get to implementing it eventually?

@cabanier

cabanier commented Dec 2, 2021

Copy link
Copy Markdown
Member

The more I think about it, moving lighter from a compositing mode to a blend mode is cheating. I'll think of a better way to do this.

Yes, don't make it into a blend mode. It doesn't solve the issue with clamping.
Are you looking into making it apply to CSS or only canvas?

@cabanier maybe you have thoughts here? Right now globalCompositingOperation has both blend modes and composite modes. Maybe the right thing to do is make mix-blend-mode the same. However, Chrome is only interested in implementing lighter right now.

The plan (from 8 years ago?) was to make another css property for compositing but that never happened.
I was never sure about the compositing part because it feels too low level.

Should the spec change be limited to what Chrome's currently prepared to implement, or converge with globalCompositingOperation hoping that browsers get to implementing it eventually?

Is Chrome looking into implementing cross-fade in css?

@jakearchibald

Copy link
Copy Markdown
Contributor Author

Are you looking into making it apply to CSS or only canvas?

The purpose of this PR is to allow developers to make an element composite using lighter. Among other things, it allows developers to cross-fade elements https://jakearchibald.com/2021/dom-cross-fade/

lighter is already supported in canvas, so the aim here is to add it to CSS.

The plan (from 8 years ago?) was to make another css property for compositing but that never happened.
I was never sure about the compositing part because it feels too low level.

Yeah, some combinations of blend & composite don't seem to make much sense.

Is Chrome looking into implementing cross-fade in css?

Not cross-fade() (we already support that as -webkit-cross-fade()), but the ability to cross-fade two DOM elements.

I've updated the PR to use 'lighter' rather than 'plus-lighter'.

@cabanier

cabanier commented Dec 2, 2021

Copy link
Copy Markdown
Member

Is Chrome looking into implementing cross-fade in css?

Not cross-fade() (we already support that as -webkit-cross-fade()), but the ability to cross-fade two DOM elements.

I've updated the PR to use 'lighter' rather than 'plus-lighter'.

If the intent is for cross-fading, making lighter a compositing mode won't work because it won't behave as expected when there's foreground or background alpha.
You either need to create a new css property that just does the cross fading or introduce compositing.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

Are you sure? I think isolation does the trick, no? Fwiw it behaves as expected in canvas. See the demos in https://jakearchibald.com/2021/dom-cross-fade/

@cabanier

cabanier commented Dec 2, 2021

Copy link
Copy Markdown
Member

Are you sure? I think isolation does the trick, no? Fwiw it behaves as expected in canvas. See the demos in https://jakearchibald.com/2021/dom-cross-fade/

Maybe it does the right thing. The alpha of the backdrop determines how much of the blend operation is applied. It's possible that this ends up doing what is expected.

@jakearchibald

jakearchibald commented Dec 3, 2021

Copy link
Copy Markdown
Contributor Author

@cabanier

cabanier commented Dec 3, 2021

Copy link
Copy Markdown
Member

I think it does. It's what Chrome uses for -webkit-cross-fade() https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/graphics/crossfade_generated_image.cc;l=73

Maybe... That code is setting a "blend mode" which is really a compositing mode.
The easiest way to check this is to do write a test page that blends in javascript on a canvas.

@khushalsagar

Copy link
Copy Markdown
Member

@cabanier I'm a bit confused about how blend-mode vs compositing-mode are supposed to be different.

There are existing blend modes which define how the source and backdrop color values should be combined (the blending step) and then how this combined result is modulated by the backdrop alpha (the compositing step). For example, the lighten blend-mode states something similar to what we want to do with lighter, the backdrop is replaced with the result of blending the 2 color values instead of any further compositing operation. Is that not a valid way to specify a mix-blend-mode?

@cabanier

Copy link
Copy Markdown
Member

A blend mode changes the color of the foreground color based on the blending formula and the alpha of the backdrop.
This changed color is then fed into the compositing mode (which is always source-over).

@khushalsagar

Copy link
Copy Markdown
Member

This changed color is then fed into the compositing mode (which is always source-over)

That's the part I'm unsure about. The text for lighten says, "The backdrop is replaced with the source where the source is lighter". Are we doing source over compositing as defined below :

co = αs x Cs + αb x Cb x (1 – αs), where

Cs = max(Cs, Cb) via the blending operation
Cb = same Cb as the blending step

@cabanier

Copy link
Copy Markdown
Member

The spec has a step by step explanation how blending is done: https://drafts.fxtf.org/compositing-1/#blending

@khushalsagar

khushalsagar commented Dec 14, 2021

Copy link
Copy Markdown
Member

Thank you. This makes it much clearer now. You're right that what we want here is "lighter" composite operator with "normal" blend mode. And there is currently no way to change the composite operator in CSS.

Would it be reasonable to include composite operators in mix-blend-mode and follow a pattern similar to globalCompositeOperation. The latter lets you specify a blend mode (in which case we default to source-over compositing) or composite operator (in which case we default to normal blending) : https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/graphics/graphics_types.cc;l=56;drc=d7044ac79851029fa1698731d9ce5c251a5939be

We can have mix-blend-mode provide the same capability as globalCompositeOperation in CSS. It's unclear why a developer shouldn't be able to specify any combination of the 2 but the syntax could be extended to permit that for both mix-blend-mode and globalCompositeOperation going forward.

@cabanier

Copy link
Copy Markdown
Member

Would it be reasonable to include composite operators in mix-blend-mode and follow a pattern similar to globalCompositeOperation.

I'm unsure. At the time, it seemed to me that these compositing modes were too primitive to expose directly which is why we punted them to later.
I think that if you want to create a certain effect with compositing, you should create that effect instead.

@khushalsagar

Copy link
Copy Markdown
Member

I think that if you want to create a certain effect with compositing, you should create that effect instead.

Hmmm, I didn't follow the alternate being proposed. Could you clarify how developers should do this instead of being able to configure the compositing of 2 elements?

A demo showing the exact use-case is here. It's using canvas to mimic the capability needed in CSS. We have 2 elements with partially identical pixels that draw on top of each other and we need to ensure that blending the identical pixels is a no-op.

@w3c w3c deleted a comment from pixlephonexl Dec 31, 2021
@w3c w3c deleted a comment from pixlephonexl Dec 31, 2021
@jakearchibald jakearchibald changed the title Add plus-lighter to mix-blend-mode and background-blend-mode Add plus-lighter and plus-darker to mix-blend-mode and background-blend-mode Jan 6, 2022
@jakearchibald jakearchibald changed the title Add plus-lighter and plus-darker to mix-blend-mode and background-blend-mode Add plus-lighter to mix-blend-mode and background-blend-mode Jan 11, 2022
@jakearchibald

Copy link
Copy Markdown
Contributor Author

This PR is ready to land. It's unclear if plus-darker should be part of this, or if plus-darker should be removed from the spec altogether. That can be addressed in another PR.

@cabanier what are the next steps?

@cabanier

Copy link
Copy Markdown
Member

@cabanier what are the next steps?

It should be simple enough to create this as an experiment in a version of chromium that you built yourself. If the elements can be blended by regular drawing (and not by the compositor), you just need to add it to skia. You could even just replace a blend mode with plus-darker.

I'm still unsure if the compositing mode will always work in the way you expect. Also, blending creates a stacking context out of the content that is blending but not the content that you are blending with. This could cause some confusion for authors.

@cabanier

Copy link
Copy Markdown
Member

I'm still unsure if the compositing mode will always work in the way you expect. Also, blending creates a stacking context out of the content that is blending but not the content that you are blending with. This could cause some confusion for authors.

Regardless, the CSSWG resolved to add it. Is there something specific about the PR that needs improvement before landing?

OK then. Not sure why you need my input if it's set in stone.

@chrishtr

Copy link
Copy Markdown
Contributor

Not set in stone, but you're the editor so we need your code review to land the PR. If there is a specific testcase you have in mind that we should check in the prototype, or a WPT example to add, we're happy to do that.

If your concern is a general one about fit for our desired use cases, that's legit but IMO the best way to find that out is by testing with partners later on, not blocking the a PR for an initial spec if we don't know why it is problematic...

@cabanier

Copy link
Copy Markdown
Member

I agree.
Great to hear that you implemented this behind a flag already. @jakearchibald, did you already experiment with the flag and determine that it satisfies your use case?
I don't see any problems adding it to background-blend-mode because that feature doesn't deal with layout and stacking contexts.
For mix-blend-mode, it's more complicated so we need examples and author feedback.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

Yep, if I take a nightly from https://download-chromium.appspot.com/ and use the --enable-blink-features=CSSMixBlendModePlusLighter flag, the tests pass, as does the practical example at https://jakearchibald.com/2021/dom-cross-fade/#update-it-already-works-in-safari

@jakearchibald jakearchibald changed the title Add plus-lighter to mix-blend-mode and background-blend-mode [css-compositing-2] Add plus-lighter to mix-blend-mode and background-blend-mode Jan 12, 2022
@chrishtr

Copy link
Copy Markdown
Contributor

Furthermore, the layout effects of causing a stacking context are not a problem for the desired use cases.

Rik, why do you think more author feedback is needed for that case? mix-blend-mode already causes a stacking context, and that is necessary in order for it to be well-defined and implementable in general. plus-lighter is no exception.

@cabanier

Copy link
Copy Markdown
Member

Rik, why do you think more author feedback is needed for that case? mix-blend-mode already causes a stacking context, and that is necessary in order for it to be well-defined and implementable in general. plus-lighter is no exception.

The stated use case for this feature was: This allows any two elements to be cross-faded together.
I'm assuming that this means that this feature is supposed to fade between 2 images or text blocks which mix-blend-mode doesn't do. Can you post an example where 2 text blocks are faded that doesn't need a lot more additional CSS to line things up? It's possible that I'm overlooking something...

@chrishtr

chrishtr commented Jan 12, 2022

Copy link
Copy Markdown
Contributor

The example here is such a case: https://jakearchibald.com/2021/dom-cross-fade/#update-it-already-works-in-safari.

It has two sibling SVG elements with mix-blend-mode: plus-lighter and a containing element with isolation:isolate (which causes a stacking context).

The CSS to do this is not a lot - just the one containing div and 3 lines of CSS.

@cabanier

cabanier commented Jan 12, 2022

Copy link
Copy Markdown
Member

I was thinking of an HTML example. Couldn't you could already do this in SVG with feComposite?

@chrishtr

Copy link
Copy Markdown
Contributor

What we need is a method that works for all kinds of HTML content, including in particular for replaced elements containing snapshot images from a current or previous document. I don't think SVG is general enough to cover all such cases.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

@cabanier I only used SVG in that example because the scaling worked well for the article. I can make a demo using DOM elements if I really need to, but SVG elements are also DOM elements so that demo does the job IMO

@cabanier

Copy link
Copy Markdown
Member

@jakearchibald Yes, please make an example with HTML elements to see how most people are going to use this feature.

@mstange

mstange commented Jan 12, 2022

Copy link
Copy Markdown

Can you post an example where 2 text blocks are faded that doesn't need a lot more additional CSS to line things up?

With CSS Grid you can line up "stacked" elements very easily. In this example, all direct child elements of .wrapper will occupy the same area:

.wrapper { display: grid; }
.wrapper > * { grid-area: 1 / 1 / 2 / 2; }

@jakearchibald

Copy link
Copy Markdown
Contributor Author

Yep! That's the pattern I recommended in https://youtu.be/PYSOnC2CrD8 too

@jakearchibald

Copy link
Copy Markdown
Contributor Author

@cabanier https://static-misc-3.glitch.me/composite-test/mix-blend-mode-dom.html - here's a demo using only DOM elements.

Of course, the cross-fade is likely to be an animation triggered by a click, but the slider used on this page makes it easier to see the difference between the default compositing method (source-over) and plus-lighter.

Summary of material relating to this:

@cabanier

Copy link
Copy Markdown
Member

Nice! I didn't know you could do it with grid :-)

@jakearchibald

Copy link
Copy Markdown
Contributor Author

@cabanier is this good to land then? Anything else I need to do?

@cabanier

Copy link
Copy Markdown
Member

@cabanier is this good to land then? Anything else I need to do?

This looks good!
I do have a question about the implementation. Was it correctly implemented so the timing of the plus-ligher operation is not dependent on color or alpha value?

@jakearchibald

Copy link
Copy Markdown
Contributor Author

I'm not sure what you mean by 'timing', can you explain a bit?

@cabanier

cabanier commented Jan 14, 2022

Copy link
Copy Markdown
Member

After we shipped this feature with blend modes that weren't susceptible to timing, someone on the skia team refactored the formulas which enabled an attack. See https://arstechnica.com/information-technology/2018/05/chrome-and-firefox-leaks-let-sites-steal-visitors-facebook-names-profile-pics/
You need to make sure that plus-lighter executes the same regardless of its input. Since there's a min(..., ...), there might be an if block in the implementation which would introduce a timing attack.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

Gotcha. We'll double check that.

That shouldn't block the spec landing though, right? It's an implementation detail.

@cabanier

Copy link
Copy Markdown
Member

Gotcha. We'll double check that.

That shouldn't block the spec landing though, right? It's an implementation detail.

Yes, the spec already specifies this but it's easily overlooked.

@jakearchibald

Copy link
Copy Markdown
Contributor Author

Can the spec and tests land now, then?

@cabanier cabanier merged commit afdf079 into w3c:main Jan 17, 2022
@cabanier

Copy link
Copy Markdown
Member

Yes :-)

@jakearchibald jakearchibald deleted the plus-lighter-mix-mode branch January 17, 2022 20:19
@jakearchibald

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing and landing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants