Skip to content

Implement 3D shadows in the GL Compatibility renderer#77496

Merged
YuriSizov merged 1 commit into
godotengine:masterfrom
clayjohn:GLES3-shadows
Sep 28, 2023
Merged

Implement 3D shadows in the GL Compatibility renderer#77496
YuriSizov merged 1 commit into
godotengine:masterfrom
clayjohn:GLES3-shadows

Conversation

@clayjohn

@clayjohn clayjohn commented May 26, 2023

Copy link
Copy Markdown
Member

Fixes: #66596

Implementation of 3D shadows for the Compatibility renderer

Implementation based on godotengine/godot-proposals#3959

Main approach

Particularly this uses a multipass approach to lighting so that we can benefit from prefetch when using hard shadows. In practice this means:

  1. All lights without shadows are rendered during base pass as well as one light with shadows
  2. Each additional light with a shadow requires drawing the object again with additive blending
  3. Lights with shadows are blended in sRGB space instead of linear, so the colors blow out very quickly (i.e. lights with shadows look like they do in the Godot 3.x GLES2 renderer) (Lights without shadows look normal)

Comments

There is still one outstanding bug where shadow atlas textures are getting out of date with multiple shadowed lights in the scene.

Pictures:

Linear vs sRGB blending

Lighting in linear space (no shadows)
Screenshot from 2023-05-25 20-49-27

Lighting in sRGB space
Screenshot from 2023-05-25 20-51-26

Lighting in sRGB space with reduced brightness and attenuation
Screenshot from 2023-05-25 20-51-08

Compatibility vs Forward+

Compatibility
Screenshot from 2023-09-27 16-59-51

Forward+
Screenshot from 2023-09-27 16-25-52

@Calinou

Calinou commented May 26, 2023

Copy link
Copy Markdown
Member

My gut tells me the performance gains from doing multipass in terms of prefetchare outweighed by the performance losses from having to render each object multiple times.

In practice, we can't expect every mobile project to be using fully baked lightmaps (either for package size reasons or because levels are procedurally generated), so I think improving performance of rendering multiple real-time lights with shadows is key. Even if you use fully baked lightmaps, lights have to be manually hidden after baking so they don't take up rendering time, and this means dynamic objects won't receive direct light (dynamic capture currently doesn't have an option to store direct light).

We also currently lack lightmap support in the Compatibility rendering method anyway.

  1. Prefetch only applies to texture samples taken directly from interpolaters, so all the PCF5 and PCF13 samples don't benefit.

Are there any alternative approaches that could be used, such as dithering? While Godot currently defaults to hard shadows on mobile, I feel like it should be possible to have something better on mobile in 2023.

@piramidamsc

Copy link
Copy Markdown

Directional Light doesn't work properly for me, I'm using your custom godot 4.1 dev build

@Calinou

Calinou commented Jun 7, 2023

Copy link
Copy Markdown
Member

I've created testing projects to compare shadow rendering performance in 3.x and 4.x:

Here are the results on my machine (Linux, GeForce RTX 4090). All tests are done in 3840×2160:

3.5.2 GLES3 3.5.2 GLES2 4.0.3 Forward+ 4.0.3 Mobile 4.0.3 Compatibility
(no shadows)
4.x Compatibility
(this PR)
1679 FPS
(0.60 mspf)
2500 FPS
(0.40 mspf)
1245 FPS
(0.80 mspf)
1249 FPS
(0.80 mspf)
2889 FPS
(0.35 mspf)
1728 FPS
(0.58 mspf)1

This error message is frequently spammed in the editor:

ERROR: Condition "!sli" is true. Continuing.
   at: _shadow_atlas_find_shadow (drivers/gles3/storage/light_storage.cpp:946)

Footnotes

  1. Rendering is broken when the DirectionalLight has shadows enabled. This issue does not occur in the editor, regardless of whether the preview environment is enabled or not. image

@filipworksdev

This comment was marked as off-topic.

@Calinou

Calinou commented Jul 9, 2023

Copy link
Copy Markdown
Member

Does this fix the longstanding issue where shadows ignore shader parameter POINT_SIZE for point meshes Mesh.PRIMITIVE_POINTS?

Please open a separate issue with a minimal reproduction project attached.

@reduz

reduz commented Aug 14, 2023

Copy link
Copy Markdown
Member

@clayjohn

In order to avoid lights flickering between Linear and sRGB blending, shadowed lights always have to be in a separate pass. In godotengine/godot-proposals#3959 reduz assumed we could render one positional and one directional light in the base pass, but in practice this looks really bad.

I can see why. That said, I would still try to, at least, do the directional light in a single pass merged with the non shadowed lights. Directional + non shadowed lights is the most common way to do lighting on mobile, so this as single pass is ideal.

I know this would break if you have more than one directional light, but we could have a setting allow_multiple_directional_lights on compatibility that works as you describe only if user enables it.

Also shadow filter is ok and expected to be non-pcf on mobile by default (only enable linear interpolation so it does hardware accelerated PCF4, which is per spec on GLES3).

@reduz

reduz commented Aug 14, 2023

Copy link
Copy Markdown
Member

@Calinou I am surprised performance is not similar to Godot GLES2 to be honest. We really should aim for as closest as that if possible.

@clayjohn

Copy link
Copy Markdown
Member Author

@clayjohn

In order to avoid lights flickering between Linear and sRGB blending, shadowed lights always have to be in a separate pass. In godotengine/godot-proposals#3959 reduz assumed we could render one positional and one directional light in the base pass, but in practice this looks really bad.

I can see why. That said, I would still try to, at least, do the directional light in a single pass merged with the non shadowed lights. Directional + non shadowed lights is the most common way to do lighting on mobile, so this as single pass is ideal.

I know this would break if you have more than one directional light, but we could have a setting allow_multiple_directional_lights on compatibility that works as you describe only if user enables it.

Also shadow filter is ok and expected to be non-pcf on mobile by default (only enable linear interpolation so it does hardware accelerated PCF4, which is per spec on GLES3).

I already have a plan. I will move the calculation of the additional lights to after the tonemapping step in the shader. That way, we can have the first light with shadows calculated in the original pass, but still in sRGB space. Should be enough to improve performance a little bit and ensure we get the best performance possible when using only a few lights.

Comment thread drivers/gles3/rasterizer_scene_gles3.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated
Comment thread drivers/gles3/storage/light_storage.cpp Outdated

@AThousandShips AThousandShips left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad had missed some in my first pass

Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
Comment thread drivers/gles3/storage/light_storage.h Outdated
@clayjohn clayjohn force-pushed the GLES3-shadows branch 6 times, most recently from 582692c to fa026d3 Compare September 27, 2023 15:43

@reduz reduz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely outstanding work!!

@reduz

reduz commented Sep 28, 2023

Copy link
Copy Markdown
Member

I still wonder why the GLES2 version of Godot 3 is faster, what do we do different there?

@BastiaanOlij

Copy link
Copy Markdown
Contributor

@reduz maybe it's faster because we don't do any of the HDR lighting calculations? All lighting is kept LDR while here we do some conversions to/from HDR while we write out LDR?

@BastiaanOlij BastiaanOlij left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll find some time to properly test this in VR as soon as I can but as far as I can tell this looks sublime! Amazing stuff Clay!

@clayjohn

Copy link
Copy Markdown
Member Author

I still wonder why the GLES2 version of Godot 3 is faster, what do we do different there?

Those measurements were taken with the old version of this PR. Back then we did an additional pass per light with shadows. Now the base pass includes one shadowed light so it is much faster

@Calinou

Calinou commented Sep 28, 2023

Copy link
Copy Markdown
Member

I've run benchmarks again on the latest revision of this PR (rebased against master), using the same hardware, resolution and testing projects as my prior test:

3.5.2 GLES3 3.5.2 GLES2 master Forward+ master Mobile master Compatibility
(no shadows)
4.x Compatibility
(this PR)
1705 FPS (0.59 mspf) 2550 FPS (0.39 mspf) 1249 FPS (0.80 mspf) 1247 FPS (0.80 mspf) 2890 FPS (0.35 mspf) 1828 FPS (0.55 mspf)

Directional shadow rendering still appears broken in the running project, even though it works in the editor:

Screenshot_20230928_143329 webp

In comparison, this is how the scene looks like in Forward+ (disregard the FPS count here, I resized my window quickly):

Screenshot_20230928_143932 webp

Visual comparison in the editor:

Forward+ Mobile Compatibility
Screenshot_20230928_143726 webp Screenshot_20230928_143744 webp Screenshot_20230928_143751 webp

@YuriSizov

Copy link
Copy Markdown
Contributor

Woohoo! Thanks!

@clayjohn

Copy link
Copy Markdown
Member Author

Just noting a comment from the top post:

There is still one outstanding bug where shadow atlas textures are getting out of date with multiple shadowed lights in the scene.

I'll investigate that outstanding bug and any new reports ASAP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

OpenGL: Light3D shadow rendering is not reimplemented yet

8 participants