Skip to content

[WIP] Vulkan: Use of hard-coded MSAA value (e.g. in mesh preview) causes GPU hangs #53940

Description

@follower

Godot version

Godot Engine v4.0.dev.20211015.official.f113dc986 (and earlier)

System information

Linux, Elementary OS 5.1, Intel(R) HD Graphics 4400 (HSW GT2), Vulkan API 1.2.182

Issue description

Summary:

Viewing a Mesh resource (seemingly of any subtype but most recently tested with BoxMesh) in the inspector results in a GPU hang.

Details:

  • Initially the GPU hang issue was observed to occur when opening/editing a MeshInstance3D that contained a mesh (or when creating one) but the specific cause was narrowed down to the rendering of the preview.

    (This means that if the "Node" tab is top-most--rather than the "Inspector" tab--it is possible to avoid the variant of the hang that prevents a newly created mesh from being saved when first created.)

  • It also turns out the underlying issue was the reason why selecting a MSAA setting of 2X (Viewport::MSAA_2X) for the project settings also led to a GPU hang.

My impression is that this specific issue may be GPU/driver specific however I have not isolated the lowest level of bug detail yet to be able to confirm this.

Specific cause of hang

The hang occurs as a result of this code:

viewport->set_msaa(Viewport::MSAA_2X);

When running under gdb, if the line is skipped (resulting in a default MSAA value of Viewport::MSAA_DISABLED) or the value is patched (to be the value of Viewport::MSAA_DISABLED) then the GPU hang does not occur.

(see also:

godot/scene/main/viewport.h

Lines 103 to 110 in 58aa020

enum MSAA {
MSAA_DISABLED,
MSAA_2X,
MSAA_4X,
MSAA_8X,
// 16x MSAA is not supported due to its high cost and driver bugs.
MSAA_MAX
};
)

Note: The issue does not occur for the Material Editor, presumably because it uses a value of Viewport::MSAA_4X (which for whatever reason doesn't appear to be affected):

viewport->set_msaa(Viewport::MSAA_4X);

Vulkan Validation output

When run with Vulkan validation layers enabled the output (in part) includes the message:

ERROR: VALIDATION - Message Id Number: 1607255571 | Message Id Name: VUID-VkImageCreateInfo-samples-02258
        Validation Error: [ VUID-VkImageCreateInfo-samples-02258 ] Object 0: handle = 0xabcee50, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x5fccc613 |vkCreateImage(): samples VK_SAMPLE_COUNT_2_BIT is not supported by format 0x0000000D. The Vulkan spec states: samples must be a bit value that is set in imageCreateSampleCounts (as defined in Image Creation Limits) (https://vulkan.lunarg.com/doc/view/1.2.182.0/linux/1.2-extensions/vkspec.html#VUID-VkImageCreateInfo-samples-02258)
        Objects - 1
                Object[0] - VK_OBJECT_TYPE_DEVICE, Handle 180153936
   at: _debug_messenger_callback (drivers/vulkan/vulkan_context.cpp:157)

Note particularly the portion which reads samples VK_SAMPLE_COUNT_2_BIT is not supported by format 0x0000000D.

That said, the format 0x0000000D does look a bit suspicious to me because AFAICT it doesn't actually match with a valid format value--which, based on some debugging I've performed, makes me wonder if something unexpected is happening at some point as the parameters are being processed.

Additional code context

The initial call to Viewport::set_msaa() ends up here:

godot/scene/main/viewport.cpp

Lines 2777 to 2784 in 56078cc

void Viewport::set_msaa(MSAA p_msaa) {
ERR_FAIL_INDEX(p_msaa, MSAA_MAX);
if (msaa == p_msaa) {
return;
}
msaa = p_msaa;
RS::get_singleton()->viewport_set_msaa(viewport, RS::ViewportMSAA(p_msaa));
}

Which then eventually seems to find its way to RendererViewport::viewport_set_msaa() here:

void RendererViewport::viewport_set_msaa(RID p_viewport, RS::ViewportMSAA p_msaa) {
Viewport *viewport = viewport_owner.get_or_null(p_viewport);
ERR_FAIL_COND(!viewport);
if (viewport->msaa == p_msaa) {
return;
}
viewport->msaa = p_msaa;
_configure_3d_render_buffers(viewport);
}

(See also:

Which then heads to RendererViewport::_configure_3d_render_buffers() here:

void RendererViewport::_configure_3d_render_buffers(Viewport *p_viewport) {
if (p_viewport->render_buffers.is_valid()) {
if (p_viewport->size.width == 0 || p_viewport->size.height == 0) {
RSG::scene->free(p_viewport->render_buffers);
p_viewport->render_buffers = RID();
} else {
float scale_3d = p_viewport->scale_3d;
if (Engine::get_singleton()->is_editor_hint()) {
// Ignore the 3D viewport render scaling inside of the editor.
// The Half Resolution 3D editor viewport option should be used instead.
scale_3d = 1.0;
}
// Clamp 3D rendering resolution to reasonable values supported on most hardware.
// This prevents freezing the engine or outright crashing on lower-end GPUs.
const int width = CLAMP(p_viewport->size.width * scale_3d, 1, 16384);
const int height = CLAMP(p_viewport->size.height * scale_3d, 1, 16384);
RSG::scene->render_buffers_configure(p_viewport->render_buffers, p_viewport->render_target, width, height, p_viewport->msaa, p_viewport->screen_space_aa, p_viewport->use_debanding, p_viewport->get_view_count());
}
}
}

[TODO: Add additional details.]

Steps to reproduce

[TODO]

Minimal reproduction project

[TODO]

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions