It would be good to get some ideas about the impact on performance

Performance is similar to viewport debanding when there's no overdraw, but it'll be lower if there's overdraw in rendering methods that don't use a depth prepass. The difference is probably not too noticeable at resolutions that make sense for the target hardware, as this algorithm is one of the fastest ones out there.

Actually, that brings an interesting topic to mind, how much will debanding effect game streaming..

Lossy compression will generally destroy any form of dithering pattern (especially subtle ones), so streaming shouldn't be noticeably affected by debanding.

Is there a use-case for toggling debanding on a single material but not others?

Yes, flat-color materials typically need it more than materials with detailed textures. However, since textures are sometimes used for materials that look flat-color (but are still using a texture), we can't enable debanding only on untextured materials.

That said, we can probably have a project setting to force debanding on all materials regardless of the use_debanding render mode (similar to how forced vertex shading works).

Yes, flat-color materials typically need it more than materials with detailed textures. However, since textures are sometimes used for materials that look flat-color (but are still using a texture), we can't enable debanding only on untextured materials.

Would there be harm in having debanding enabled on textured materials even when not needed? Is your concern for the performance overhead?

I'm trying to figure out if there is a use-case for a user to go and enable this project setting on materials individually. From what I can tell it makes more sense as an all-or-nothing setting where it applies to all materials or none.

Is your concern for the performance overhead?

Yes, it may be non-negligible if you have overdraw in your scene when using the mobile renderer. If this isn't important for the target hardware, the project setting to force debanding on all materials would address this.

I've looked into converting this PR into a project setting and have pushed an additional commit. However, it doesn't appear to work and am not sure how to get it working in Compatibility as well. I see the GLES3 scene shader has a spec_constant emulation still.

However, should we be using a different shader version or specialization constant here? It's a setting that won't be toggled during gameplay after all, so I'm not sure if we should use up space in the specialization constant map for this (in Forward+/Mobile, that is).

However, should we be using a different shader version or specialization constant here? It's a setting that won't be toggled during gameplay after all, so I'm not sure if we should use up space in the specialization constant map for this (in Forward+/Mobile, that is).

In the RD renderers we have a set of spec constants that don't change frequently. I would use spec constants and then group it with those.

The reason this is preferable is so that users can change the setting for all materials without restarting the editor.

If you were to make a shader version then you wouldn't have an easy way to invalidate all shaders and update (as you only set the version at shader compile time).

Here is a stress test. Please consider changing to dither.rgb - 0.8 for better results, because otherwise when using blend_add materials the dithering patter starts to stand out on overlapping meshes.

Original scene, no dither:

Different subtraction values (I suggest dither.rgb - 0.8):

Currently, with dither.rgb - 0.5 the pattern is visible even on absolutely black overlapping materials:

Here is a stress test. Please consider changing to dither.rgb - 0.8 for better results, because otherwise when using blend_add materials the dithering patter starts to stand out on overlapping meshes.

We should probably disable the dither altogether for non-opaque surfaces. Shifting the dither by 0-8 will introduce stronger dithering artifacts for brighter scenes

Here is a stress test. Please consider changing to dither.rgb - 0.8 for better results, because otherwise when using blend_add materials the dithering patter starts to stand out on overlapping meshes.

We should probably disable the dither altogether for non-opaque surfaces. Shifting the dither by 0-8 will introduce stronger dithering artifacts for brighter scenes

Is that the case for blend_add, when ALPHA is not involved? What kind of solution the above scene would benefit from if dithering is disabled?
And also, can this sort of dithering be done in .gdshader on demand then?
Edit: never mind, I can just implement it in .gdshader then.

Is that the case for blend_add, when ALPHA is not involved? What kind of solution the above scene would benefit from if dithering is disabled? And also, can this sort of dithering be done in .gdshader on demand then? Edit: never mind, I can just implement it in .gdshader then.

Anything with blend_add is non-opaque. So yes, it would apply even when ALPHA is not involved. The problem is the dithering takes place in screen space. So if you add a material multiple times, then the dither is applied the same way to the same pixel and makes it much stronger.

The other alternative would be to rotate the dithering samples per material to ensure that they don't line up. That might be possible, but it would make the approach much more complex.

The other alternative would be to rotate the dithering samples per material to ensure that they don't line up. That might be possible, but it would make the approach much more complex.

This is something I toyed with for distance fade using INSTANCE_ID at some point (but I didn't open a PR). If it's a simple random-ish translation offset, it's cheap but still quite effective in my experience.

Also, just in case, I suggest to test debanding constants in the project shader for the best pattern look across all colors.
Also I still suggest to check the subtracted brightness value, or possibly make it an environment variable. And not opt-out transparent materials altogether. For instance: https://github.com/roalyr/GD4Lancer/blob/main/Assets/Shaders/Clouds/Cloud_cull_front.gdshader

I would like to spend some time deeply reviewing this PR, but I won't have time until I finish up work on the HDR output PR. That said, a couple of things come to mind at an initial glance at the code changes that I would like to quickly point out sooner rather than later:

1. This screen space dithering function was designed to be applied as the final step before quantization from floating point values to integer values. The / 255.0 part of the function is specific to an 8-bit buffer and it should likely be changed to / 1023.0 when the quantization target is a 10-bit buffer. (This probably means / 255.0 for Compatibility and / 1023.0 for Mobile.)
2. Because of point 1, I expect that having this applied in the Forward+ rendering method will produce the same issue described in Color correction and debanding function incorrectly when HDR 2D is enabled #107730, but I haven't tested it yet. I believe the solution is to simply disable material dithering for the Forward+ rendering method and only have this dithering applied as the final step before quantization (as in, only for the Compatibility and Mobile rendering methods).
3. I wrote some more notes on the subject here and here. Specifically, it's possible to implement a debanding dither that can be applied to a linear encoding, like described by Timothy Lottes, but I suspect that this still must be applied in the specific order of 1) special linear dither, 2) inverse EOTF, 3) quantization (??? I haven't deeply looked into this approach, so I might be wrong???). I don't think these exact steps apply here...
4. The original dither function from Valve was designed to be applied with a time parameter and only applied with a 37.5% strength. The strength used in Godot's debanding has historically been 100% with no time parameter. These choices should probably be revisited (maybe a different debanding strength makes sense). For example, to apply a 37.5% debanding strength, you simply replace / 255.0 with * 0.375 / 255.0 or for this scenario of a 10-bit quantization replace / 1023.0 with * 0.375 / 1023.0. Maybe an even larger dither strength above 1.0 could be considered.
5. I am adding a number of comments to the existing dither function in the codebase as a part of my Always perform color correction and debanding on nonlinear sRGB values. #107782 bugfix PR. These comments should probably be proliferated to reduce accidental misuse of the dithering function.

Again, sorry, I haven't actually tested this out, but I wanted to get the theory behind this concept voiced sooner rather than later. I'll revisit this in the future weeks... (feel free to mention me if you don't hear back!)

(Edit: I removed a possibly erroneous comment about Compatibility)

I'm currently working on improving this debanding PR, using some of the ideas from my previous comment.

The effect is subtle and best viewed at full resolution in a dark surround, but here's what the before and after looks for Mobile (notice how material debanding with 10-bit aligned dither on mobile is still effective at debanding while producing less noise):

8-bit debanding (this PR)	10 bit debanding

The Compatibility debanding is proving to be hard to improve on for the case of additive lights. I also think I need to get a better understanding of the buffer formats that come into play...

The effect is subtle and best viewed at full resolution in a dark surround, but here's what the before and after looks for Mobile (notice how material debanding with 10-bit aligned dither on mobile is still effective at debanding while producing less noise):

When viewing images at 100% zoom in a new tab, I spot some "rings" around the lights in the 10-bit debanding that aren't there in the 8-bit debanding. Is that normal, or something due to my monitor's calibration?

The effect is subtle and best viewed at full resolution in a dark surround, but here's what the before and after looks for Mobile (notice how material debanding with 10-bit aligned dither on mobile is still effective at debanding while producing less noise):

When viewing images at 100% zoom in a new tab, I spot some "rings" around the lights in the 10-bit debanding that aren't there in the 8-bit debanding. Is that normal, or something due to my monitor's calibration?

Ah, yes, the banding is a bit more visible around the yellow light with the 10-bit based debanding, while the 8-bit is more noisy in the shadows. Good catch. I'll look a bit more into it. Personally, I prefer having a the 10-bit image that has less noise overall, but maybe others would prefer having more noise and even less banding... (Both images are a massive improvement over no debanding, of course!)

Personally, I prefer having a the 10-bit image that has less noise overall, but maybe others would prefer having more noise and even less banding... (Both images are a massive improvement over no debanding, of course!)

I tend to be more bothered by ringing than noise personally, especially on a display with higher pixel density where the noise becomes less noticeable, but not the ringing.

I'm currently working on improving this debanding PR, using some of the ideas from my previous comment.

The effect is subtle and best viewed at full resolution in a dark surround, but here's what the before and after looks for Mobile (notice how material debanding with 10-bit aligned dither on mobile is still effective at debanding while producing less noise):
8-bit debanding (this PR) 10 bit debanding

The Compatibility debanding is proving to be hard to improve on for the case of additive lights. I also think I need to get a better understanding of the buffer formats that come into play...

10 bit looks to have less noise and less banding to me. Both look good

Ah, yes, the banding is a bit more visible around the yellow light with the 10-bit based debanding, while the 8-bit is more noisy in the shadows. Good catch. I'll look a bit more into it. Personally, I prefer having a the 10-bit image that has less noise overall, but maybe others would prefer having more noise and even less banding... (Both images are a massive improvement over no debanding, of course!)

Whoops, I was looking at the images in my web browser at 100%, which is actually 125% because my OS has everything scaled to 125%. When actually view these images at 100%, I see more banding with the "8-bit debanding" image. I see banding in both, but it's noticeably worse to me in the "8-bit". This is my attempt at highlighting the sort of banding I see with the "8-bit debanding" style of the current state of this PR:

It might actually not be banding that I'm seeing, but instead the dither pattern, as it seems to cover the most of the image.

...So I looked into this and indeed, it seems to be the dithering pattern that I'm seeing:

In the screenshot I posted, it's using both material and viewport debanding, so I made a tool for finding an offset) that works well with the base dithering pattern. In this image the top and bottom bars are the original colour, the left is the base dither pattern, the right is the offset dither pattern, and the middle is the left and right added to each other (in linear space):

(The dithered parts look darker in this example, but this isn't an issue for viewport debanding because the "darkening" pixels have more perceptual weight than the "brightening" pixels since I'm doing this blend in linear space. It's possible some brightening is happening with the linear space debanding of the material debanding... But this would be very minor for most values except the darkest.)

End result is something that has less noise than anything else I've seen for this scenario of Mobile material + viewport debanding:

At this point, I don't think I'm even fighting with debanding anymore... Instead, I think the main issues are relating to a Moiré pattern created by the two dithers, but even then, it's a huge strain on my eyes to even see this stuff with my monitor. 😵‍💫

All that to say, I'm concerned about increasing the dither strength any more because the dither pattern isn't a perfect silver bullet and, at least to me, I can see the dither pattern's diagonal lines appear on my two displays when it is applied on the incorrect space with only an 8-bit alignment instead of 10...

I'll keep chipping away at this to see if I can get some better behaviour from the Compatibility rendering method, but at least what this PR currently has for Compatibility debanding works a whole lot better than nothing!

I've mostly finished work on an second iteration of this PR that reduces what I believe is unnecessary noise by aligning the debanding to final float-to-integer quantization: https://github.com/allenwp/godot/tree/add-spatial-shader-debanding

In my branch, I did the following:

• Rebased to current master
• Corrected Mobile material debanding to be applied as the last step before quantization from floating point to integer values and corrected the scale to align to 10-bit integer values
• Disabled material debanding entirely when using the Forward+ rendering method
  • Because Forward+ uses floating point buffers for rendering materials, there is never a need for material debanding(??) Please correct me if I'm wrong and you know of a case where Forward+ needs this sort of material debanding.
• Consolidated the project setting for use_debanding so that it affects both viewport and material debanding
  • Viewport and material debanding can still be individually controlled through scripting
  • I don't see any issue with combining material and viewport debanding on the Mobile rendering method. I think most users would want this behaviour and if they don't, they can turn it off at run-time.
• Updated documentation, including a simplified project setting description that simply states that the dithering filter is applied at different steps of the rendering process depending on the rendering method and HDR 2D setting. Details about when viewport/material debanding are available are included in their documentation.

And in a second commit I did this:

• Corrected Compatibility material debanding to be applied as the last step before quantization from floating point to integer values*

After all of this, I think the result is a very simple, easy to use feature that, from a layperson's perspective, simply "fixes the bug where debanding doesn't work on the Mobile and Compatibility rendering methods".

* There is one remaining item that could be improved: the Compatibility material debanding method adds brightness to the scene after the change in my second commit. In this way, my proposed approach for Compatibility simply doesn't work. I think it's good to merge with Calinou's approach and improve Compatibility material debanding at some point in the future. When I try and understand this PR's current approach, I think the idea is to add a lot of extra noise in the first base pass and then add only add smaller amounts of noise with each additive light pass. This way, the extra lights don't make the debanding increase the brightness of the scene. I think the reason my version increases brightness is because the specific offsets for each additive pass results in a dithering pattern that adds brightness overall, but I'm not too sure just yet.

Anyway, I've kept my failed attempt for Compatibility as a separate commit in my branch so the rest can be merged into this PR if desired. @Calinou, when you have a chance, you're welcome to review and merge my version (without the failed Compatibility commit). Or I can make a new PR with you as a co-author if that's preferable!

I expect I won't have time to work on the Compatibility material debanding for the next while, so don't hold up on me figuring out a better solution for that one.

@Calinou, when you have a chance, you're welcome to review and merge my version (without the failed Compatibility commit). Or I can make a new PR with you as a co-author if that's preferable!

Feel free to open a new PR with me as a co-author. I think it's better to proceed this way, so you can make further changes more easily.

I have opened a new PR that is a simplified and trimmed down version of the material debanding introduced by this PR for the Mobile renderer. Please give my new PR a test, as I hope it will be merged in Godot 4.6: #109084 (Download the editor build artifact from the checks page.)

@roalyr Would you be interested in testing my new PR #109084 to confirm that there aren't any substantial issues with transparency that remain? I can't easily reproduce the issue that you described, so I get the feeling this was fixed at some point.

@allenwp I don't use Godot4 for a while, switched to Godot3 + GLES2 for my project, so I don't really have a suitable test subject anymore.

Rebased and tested again, it works as expected. See OP for updated screenshots.

This PR now only features debanding support in Compatibility, since Mobile debanding was implemented in #109084.

I've tested various scenarios and they all work: lights without shadows, lights with shadows, lightmaps. The noise level is kept to a minimum, although it will be more noticeable when you have multiple overlapping lights with shadows enabled (due to the use of multipass rendering). That said, if you have a lot of overlapping lights with shadows enabled, you're likely to run into performance issues in the first place given the multipass approach used in Compatibility for these lights.

Thanks for the rebase!

This debanding approach should never be applied when the shader is writing to a floating point buffer; it should only be applied when writing to an integer buffer (with /255 for an 8-bit buffer and /1023 for a 10-bit buffer).

With the current state of this PR, enabling HDR 2D switches the buffer formats to floating point buffers, but debanding is still applied. In this scenario debanding should not be applied. This should be handled per-viewport, given that different viewports can have different HDR 2D settings.

I'm not sure it is worth doing in the compatibility renderer. No one has complained about banding. In fact, everyone who has complained has pointed out that the compatibility renderer works great.

So ultimately, I'm not sure the extra complexity is worth adding

I would die to have this in compatibility

for VR banding is extremely distracting, and right now in dark scenes it's pretty atrocious

I'm not sure it is worth doing in the compatibility renderer. No one has complained about banding. In fact, everyone who has complained has pointed out that the compatibility renderer works great.

I would second the idea that banding isn't a big problem in compatibility. I use compatibility in 95% of my projects, and it has never been a problem for me (even in VR). It's the times I've tested switching my projects to the mobile renderer that I've noticed serious banding issues.

That's not to say there isn't any banding in compatibility, or that we shouldn't do this - having more options is nice :-) - just that it's never struck me as a big problem

I would die to have this in compatibility

for VR banding is extremely distracting, and right now in dark scenes it's pretty atrocious

Actually, this is an important thing to consider; when creating dark scenes in some VR headsets, debanding is very necessary. We needed this for Keep Talking and Nobody Explodes for Gear VR when the lights went out in our virtual scene; on those old HMDs, the brightness of the display was quite high, so huge amounts of banding was visible when writing to an 8-bit buffer.

That said, the solution here might be to add Viewport debanding for Compatibility HDR 2D. I believe this would resolve this issue without the need for material debanding. Viewport debanding would be added to post.glsl and would only be applied when HDR 2D is enabled.

Does this sound like it would solve your issue @michaelharmonart? Or is there a strong reason that it is impossible for you to adapt your game to use HDR 2D with the Compatibility renderer?

No one has complained about banding.

While the issue isn't as bad as in Mobile, it's still present in Compatibility to an extent at least similar to Forward+ when debanding is off (usually greater). While I didn't create an issue for it (as it wasn't a bug per se), I personally found it to be a big enough issue to be worth resolving.

It largely depends on art style; the moment you have large flat color planes (especially with fog), debanding becomes a must-have in my book. In some pixel art games, it can also be important if the texel density is low enough that the texels end up forming large enough flat planes when viewed up close. A lot of games that use Compatibility go for a simplified art style (sometimes with vertex shading to boot), where banding is more likely to be seen.

Or is there a strong reason that it is impossible for you to adapt your game to use HDR 2D with the Compatibility renderer?

The performance cost of HDR 2D in standalone VR might be prohibitive, given the memory bandwidth constraint. It's probably fine in PCVR though.

That said, if Compatibility with HDR 2D uses a RGBAH buffer, it makes sense to implement Viewport debanding for consistency with the Mobile renderer.

Does this sound like it would solve your issue @michaelharmonart? Or is there a strong reason that it is impossible for you to adapt your game to use HDR 2D with the Compatibility renderer?

edit:

Looks like I was probably using viewport debanding with a 10 bit integer buffer last time I tested and that's why I wasn't impressed.

I'll have to give HDR 2D a try (I didn't know it was an option for 3D) and see what the performance is like.

Rebased and tested again, it works as expected.

@allenwp In master, does Compatibility use a RGBAH or RGB10A2 buffer when HDR 2D is enabled? I need to know this so I can implement viewport debanding when needed.

PS: Enabling HDR 2D in Compatibility makes the scene way too contrasty in the test project, regardless of whether glow is enabled. I don't know if this is a bug.

Rebased and tested again, it works as expected.

Although this PR is pretty magical and impressive, I lean towards keeping the Compatibility renderer simpler and not merge this Compatibility material debanding. My reasoning is that the Compatibility renderer is filled with lots of little bugs that come up due to its complexity of different configurations. Adding this extra feature will increase complexity and risk more bugs and difficulty for new developers to improve the renderer. Hopefully this PR can remain useful for those who make custom engine builds. While material debanding was extremely valuable for Mobile because it was writing linear-encoded values to a 10 bit buffer, I don't see it being so necessary when writing sRGB-encoded values to an 8 bit buffer.

I'm not sure about viewport debanding; this might be simpler...

@allenwp In master, does Compatibility use a RGBAH or RGB10A2 buffer when HDR 2D is enabled? I need to know this so I can implement viewport debanding when needed.

HDR 2D means floating point buffers the whole way through. This project demonstrates this pretty clearly: even values up to the high double-digits are preserved in well in the viewport, even though it is written with sRGB encoding by post.glsl:
demo-compat-hdr-2d.zip

Is copy.glsl the equivalent of blit.glsl for Compatibility? If so, I guess this is where viewport debanding would be applied for Compatibility with HDR 2D enabled. If not, then I'm not sure where it should be applied.

PS: Enabling HDR 2D in Compatibility makes the scene way too contrasty in the test project, regardless of whether glow is enabled. I don't know if this is a bug.

Yikes. That sure looks like a bug.

Edit: It seems like this bug only affects the editor?