Skip to content

Added UI Opacity#11716

Open
Sjael wants to merge 10 commits intobevyengine:mainfrom
Sjael:opacity
Open

Added UI Opacity#11716
Sjael wants to merge 10 commits intobevyengine:mainfrom
Sjael:opacity

Conversation

@Sjael
Copy link
Copy Markdown
Contributor

@Sjael Sjael commented Feb 5, 2024

Objective

  • Making fade-in animations by changing BackgroundColor makes child nodes pop in, since opacity is not propagated to children
  • Addresses UI Opacity #6956

Solution

  • Opacity and CalculatedOpacity tuple struct components to allow child UI nodes to inherit opacity.

Things I want feedback on:

  • Precise ordering for the system, what set it should go in
  • Should this could be made into a single component with 2 fields to avoid bundle bloat? (rather not since I'd have to rewrite away from clean change detection)
  • Where to store the system declarations (right now it's in bevy_ui/lib.rs, wanted to consult before naming a new mod)
  • Ideas for better self-explanatory example

Changelog

  • Added Opacity and CalculatedOpacity
  • Added OpacityBundle
  • Changed NodeBundle / TextBundle / ImageBundle / AtlasImageBundle to have opacity components

Migration Guide

  • Constructors of NodeBundle / TextBundle / ImageBundle / AtlasImageBundle need opacity components

@Sjael Sjael changed the title Opacity UI Opacity Propagation Feb 5, 2024
@Sjael Sjael changed the title UI Opacity Propagation Added UI Opacity Feb 5, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 5, 2024

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

2 similar comments
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 5, 2024

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 5, 2024

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 5, 2024

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

@alice-i-cecile alice-i-cecile added C-Feature A new feature, making something new possible A-UI Graphical user interfaces, styles, layouts, and widgets labels Feb 5, 2024
Copy link
Copy Markdown
Contributor

@pablo-lua pablo-lua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Liked those changes but I think there is some places that can get better
One more thing
Does this work with borders and outlines? Looks like the background color isn't picked for this types in extract_uinode_borders and extract_uinode_outlines

stack_index: uinode.stack_index,
transform: transform.compute_matrix(),
color: color.0,
color: opacity * color.0,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can get very strange if opacity is > 1 or < 0, which can happen very easily, we might want to clamp somewhere on the CalculatedOpacity and warn the user to use only opacity between 0 and 1 maybe ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this fix sufficient? We could also clamp the Opacity during propagation.

impl Mul<Color> for &CalculatedOpacity {
type Output = Color;
#[inline]
fn mul(self, rhs: Color) -> Self::Output {
let alpha = rhs.a() * self.0;
*rhs.clone().set_a(alpha.clamp(0.0, 1.0))
}
}

Copy link
Copy Markdown
Contributor

@pablo-lua pablo-lua Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good for now, another possibility would be clamp in the propagate_opacity function, for example
Given the parent Opacity is 0.5, and the children opacity is 2, all the children of the given children would have a bigger opacity than the root parent, if we only clamp there.

@pablo-lua
Copy link
Copy Markdown
Contributor

what set it should go in

I think this should go before the Layout, because this is a "style" change after all, but if we want to go without ordering, looks like this will not affect too much in this case.
If Opacity is set to 0, we can skip the target in the render phase, but I'm not sure of this yet

Should this could be made into a single component with 2 fields to avoid bundle bloat

I think its okay to have this 2 opacity components, as this is used for calculating purposes
I think this should be in a new OpacityBundle (For example, we agroup Visibility and friends in VisibilityBundle, so its only fair to group these two)

Where to store the system declarations

Maybe in layout/mod.rs? All the systems used in calculating style are there, so it's only fair IMO

Ideas for better self-explanatory example

My first suggestion is using ClearColor set to white (or changing the background color of the nodes to a more lighter color so we can see the difference) in the example so we can see the opacity working, and them using the opacity in all type of ui that we can get (For example, images, buttons, text, with borders or without, etc)

And second is the fact that generally, when building ui in the examples, we don't really use helper functions, just build the ui raw, so I think we should do the same in this example? Have a look at the ui example

@Sjael
Copy link
Copy Markdown
Contributor Author

Sjael commented Feb 6, 2024

Thank you Pablo. I pushed most of those changes. I'll make those better examples a bit later tonight.

@CooCooCaCha
Copy link
Copy Markdown

UI opacity is actually a lot trickier than it might seem. Glancing at this PR, each UI node is still rendered independently right? If so, this is actually not how most UI libraries handle opacity.

On the web for example if you make a parent element transparent the parent and all its children are flattened into a single texture, then the resulting texture is rendered transparently.

What I think is necessary to do this properly is a compositor api that does the flattening.

@Sjael
Copy link
Copy Markdown
Contributor Author

Sjael commented Feb 7, 2024

What I think is necessary to do this properly is a compositor api that does the flattening.

Perhaps. How would it figure out multiple levels of opacity? As in a parent has 50% and child has 50%. Would it just set the texture it to the parent's opacity and only do this once and not go deeper?

As it is right now there isn't really a structure to opacity at all. Setting a parent node to have a BackgroundColor with transparency looks really off-putting. If it takes too long to implement what you're suggesting, this could be a good holdover considering how lightweight it is, only 2 components, 1 system. and 4 query parameters.

@CooCooCaCha
Copy link
Copy Markdown

Perhaps. How would it figure out multiple levels of opacity? As in a parent has 50% and child has 50%. Would it just set the texture it to the parent's opacity and only do this once and not go deeper?

As it is right now there isn't really a structure to opacity at all. Setting a parent node to have a BackgroundColor with transparency looks really off-putting. If it takes too long to implement what you're suggesting, this could be a good holdover considering how lightweight it is, only 2 components, 1 system. and 4 query parameters.

It's not reaaaaaally a perhaps, because that's how pretty much every major UI library works, and transparency looks strange without it. I agree, some sort of transparency support might be better than none although that's not really my call to make.

To answer your first question, rendering happens in stages based on hierarchy. Basically:

  1. Render child and all of its children to Texture 1.
  2. Render the parent and all of its children to Texture 2. When it comes time to render the child, use Texture 1 instead, and render it with 50% opacity.
  3. Render Texture 2 to the screen with 50% opacity.

BackgroundColor and opacity work slightly differently so we should be clear about the differences and a decision needs to be made if bevy should work the same. Given that, right now, bevy styling is based on web it'd be strange to not replicate this behavior.

Here's the difference: If a node has a transparent background color only that node is rendered transparently, children are rendered on top without transparency. Opacity does what I described above where the parent and children are flattened and the resulting texture is rendered transparently.

To illustrate this, I cooked up a simple demo:

Image 1: Three nested squares with no transparency. There is also text behind the green square but its blocked so you can't see it.
Screenshot 2024-02-06 202636

Image 2: Red square has 50% opacity. Notice everything gets brighter because the background is a light color. However you still can't see the text.
Screenshot 2024-02-06 202700

Image 3: Green square has 50% opacity. The blue and green squares are flattened and blended with the red. The text is now visible too.
Screenshot 2024-02-06 202714

Image 4: Green has transparent background color. Green is blended with red but blue is still fully blue. The text is hidden again.
Screenshot 2024-02-06 213632

@Sjael
Copy link
Copy Markdown
Contributor Author

Sjael commented Feb 7, 2024

I see what you mean, it makes sense that is the norm. Thank you for the example with text. Also, I was saying 'perhaps' in the context of me making that a part of this PR.

@CooCooCaCha
Copy link
Copy Markdown

I see what you mean, it makes sense that is the norm. Thank you for the example with text. Also, I was saying 'perhaps' in the context of me making that a part of this PR.

No problem. I don't really have an opinion on whether bevy should use this as a placeholder or not. Probably best to discuss in discord to see what people think.

@viridia
Copy link
Copy Markdown
Contributor

viridia commented Feb 7, 2024

@CooCooCaCha I'm in agreement with you here - the UI should be rendered into an offscreen buffer, and then the buffer should be composited onto the main view with it's own separate alpha multiplier.

I've thought a lot about this, and here's the API I would like to see:

  • Define an ECS component called CompositingBuffer which is attached to the parent element. This component will have a handle to a render target and will also have information about the size and offset of the buffered view. The size will need to be manually set, because it's hard to automatically calculate this, especially if there are drop shadows or other effects; and also, most games know perfectly well how large a dialog box is going to be, so it's actually easier to do it manually than to try and make it automatic.
  • During rendering, the parent and all of its descendants are rendered to the compositing buffer using a coordinate system specified in the component.
  • Once rendering is complete, the compositing buffer is rendered to the main window. In this phase, we can use a variety of options: opacity, scaling, custom blend modes, blurring, custom shaders and so on. And of course all these parameters can be animated.

In other words, we shouldn't just focus on opacity here, we want a more general solution. Imagine a dialog box that opens by starting out blurry and then "comes into focus" as it opens.

@Sjael Sjael closed this Feb 7, 2024
@Sjael Sjael reopened this Feb 7, 2024
@Sjael
Copy link
Copy Markdown
Contributor Author

Sjael commented Feb 7, 2024

In other words, we shouldn't just focus on opacity here, we want a more general solution. Imagine a dialog box that opens by starting out blurry and then "comes into focus" as it opens.

So you want every node to have this compositing component on it, and have a system that not just propagates opacity but other things as well, sort of like filters/effects in something like photoshop, correct?

@viridia
Copy link
Copy Markdown
Contributor

viridia commented Feb 7, 2024

So you want every node to have this compositing component on it, and have a system that not just propagates opacity but other things as well, sort of like filters/effects in something like photoshop, correct?

Not every node. Only the root node of the dialog box (or inventory panel or whatever modal ui we are talking about here). To be clear, the dialog box and all of its children get rendered into a single off-screen buffer, and then we apply various effects to that buffer as we draw it on the screen. What I'm proposing is using a special ECS component to opt-in to this behavior, so that you only pay the cost if you actually need the effect.

This is similar to how it works in browsers as well: when you apply a transform or a visual effect (such as opacity) to an element, it composites that element and all of its children to a buffer. This happens automatically, and is triggered by the presence of specific CSS properties such as opacity, transform or filter. (I know a bit about this, having written a browser during my time at Maxis).

@pablo-lua
Copy link
Copy Markdown
Contributor

Thats a pretty difficult decision we have to take with this feature in regards of design and so, but the example was very good @CooCooCaCha, thank you for clarifying this aspect!

@CooCooCaCha
Copy link
Copy Markdown

@CooCooCaCha I'm in agreement with you here - the UI should be rendered into an offscreen buffer, and then the buffer should be composited onto the main view with it's own separate alpha multiplier.

I've thought a lot about this, and here's the API I would like to see:

  • Define an ECS component called CompositingBuffer which is attached to the parent element. This component will have a handle to a render target and will also have information about the size and offset of the buffered view. The size will need to be manually set, because it's hard to automatically calculate this, especially if there are drop shadows or other effects; and also, most games know perfectly well how large a dialog box is going to be, so it's actually easier to do it manually than to try and make it automatic.
  • During rendering, the parent and all of its descendants are rendered to the compositing buffer using a coordinate system specified in the component.
  • Once rendering is complete, the compositing buffer is rendered to the main window. In this phase, we can use a variety of options: opacity, scaling, custom blend modes, blurring, custom shaders and so on. And of course all these parameters can be animated.

In other words, we shouldn't just focus on opacity here, we want a more general solution. Imagine a dialog box that opens by starting out blurry and then "comes into focus" as it opens.

This is pretty much what I was thinking. Awhile back in discord I mentioned a general-purpose compositing API and it sounds like we're on the same page in that regard. I could see this being used for all sorts of stuff including video, overlay effects, etc.

For the rendering algorithm I wonder if we could re-use composite buffers? If we pre-allocated a handful of full-screen buffers that would potentially lower memory usage and we wouldn't have to worry about synchronizing the size of the composite buffer with the size of the elements since they're all full-screen anyways.

I haven't thought about this in-depth but I think you could get away with N buffers where N is the length of the longest opacity chain in the hierarchy. For example, image a scene where elements are nested to a depth of 8, but for any given chain from root -> leaf the maximum number of transparent nodes you encounter is 4. You could potentially get away with allocating 4 full-screen buffers re-using them by rendering clusters of children to a single buffer, then going up a level and rendering another cluster of children, etc.

@Sjael
Copy link
Copy Markdown
Contributor Author

Sjael commented Feb 8, 2024

I haven't thought about this in-depth but I think you could get away with N buffers where N is the length of the longest opacity chain in the hierarchy. For example, image a scene where elements are nested to a depth of 8, but for any given chain from root -> leaf the maximum number of transparent nodes you encounter is 4. You could potentially get away with allocating 4 full-screen buffers re-using them by rendering clusters of children to a single buffer, then going up a level and rendering another cluster of children, etc.

This is exactly how I was thinking of it as well. I'm unsure if I will be able to make this a part of this PR.

@viridia
Copy link
Copy Markdown
Contributor

viridia commented Feb 8, 2024

Just to be clear, I am not in favor of the approach of modifying child opacities. While it might look OK in some cases, it will look bad in others. For example, a dialog with a textured background with buttons on top: you don't want the textured background to show through the buttons while it's fading in.

Using a compositing buffer may require more work up front, but it's more robust and general in the long run.

@CooCooCaCha If you are really clever you can get away with just two buffers: one to read, and one to write.

@pablo-lua
Copy link
Copy Markdown
Contributor

Just to be clear, I am not in favor of the approach of modifying child opacities.

We can take an approch like a component similar to Visibility and allow the user to decide somehow if they wants the opacity to be inherited by the parent?

@alice-i-cecile alice-i-cecile added S-Needs-Review Needs reviewer attention (from anyone!) to move forward D-Complex Quite challenging from either a design or technical perspective. Ask for help! S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged X-Needs-SME This type of work requires an SME to approve it. and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Jun 25, 2024
@UkoeHB
Copy link
Copy Markdown
Contributor

UkoeHB commented Sep 17, 2024

See #15256 for a proposed solution.

@BenjaminBrienen
Copy link
Copy Markdown
Contributor

@Sjael are you still interested in working on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-UI Graphical user interfaces, styles, layouts, and widgets C-Feature A new feature, making something new possible D-Complex Quite challenging from either a design or technical perspective. Ask for help! S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged X-Needs-SME This type of work requires an SME to approve it.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants