It breaks the multithreaded_compute test.

I suspect that the error is that multithreaded_compute is racing to create command buffers in the same slot:

[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(0,1,gl)  <---
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(0,2,gl)  <---
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(1,1,gl)
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(2,1,gl)
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(3,1,gl)
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(4,1,gl)
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(5,1,gl)
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is inserting CommandBufferId(6,1,gl)
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is removing CommandBufferId(0,1,gl) <---
[2024-01-26T09:34:35Z TRACE wgpu_core::storage] User is removing CommandBufferId(0,2,gl) <--- Boom.

What's happening is:

• the command buffer is used in a submission,
  • it snatches it from the registry and leaves its spot with an occupied error state.
  • the command buffer's ResourceInfo gets dropped which frees the ID too soon since the registry still has a spot for the command buffer and we haven't dropped the command buffer.
• After that a variety of things can go wrong depending on scheduling, but the one I'm observing most often is a new command buffer gets placed in the yet-to-be-dropped spot in the registry.

So we have to ensure that the ResourceInfo doesn't get dropped before we've had a chance to call command_buffer_drop and clear out the spot in the registry.

Unfortunately, trying to keep an Arc to the resource in the Destroyed variant of the registry causes CommandBuffer::from_arc_into_baked to panic because it expects to have the only arc left to the command buffer. There's a tension between this code using the reference counting to signify actual usage of the resource and the ResourceInfo info lifetime (tied to the resource itself), determining how IDs can be reused in the registry.

We could put the ResourceInfo into an arc of its own, but that would have to be done for all resources because of all of the generics involved, that's not great. We could also devise another way to decide when to reuse IDs. I would prefer it to stay close to the registry itself instead of being driven by the resource reference counts. Ideally IDs would be freed by the Gloabal::<resource>_drop functions, but for it to work well we need to remove the remaining uses of IDs during command recording otherwise one could drop/re-create a resource used by a command encoder before submit.

Another solution that doesn't require as much refactoring is to make command buffers snatchable. We probably should not use the Snatchable type for this since snatching a resource while anything wgpu-related is happening on another thread is guaranteeing lock contention. But a mutex should suffice since when we are extracting the command buffer we already expect to be the only users.

@nical with the other things you landed by now, has the situation changed much on how to fix this?

Snatchability out of the registry makes the most sense to me since this best represents what should be happening (the queue submissing takes over the lifetime of the actual resource, from a hub perspective it's already dead), but then again I'm not 100% sure I've been following correctly

I'm note sure, this is completely paged out of my brain. I'll try to rebase it and see if it breaks.

For tracking purposes: This was unblocked by #5244.