I mean, returning Ok(()) will disable the msaa_writeback and fix the crash, but I'm missing something or this will sometimes enable msaa_writeback and sometimes not, depending on the order in which the async pipeline compilation ran?

I am not a full rendering expert, but as far as I understand it, the pipeline only needs to be compiled once, but it can take time some time to compile and now that it is spawned, more than one frame ( the spawning itself can take some time, probably what happens with the example). After compilation, the pipeline will be run each frame, the protection prevent the pipeline to be run while its compilation is not yet finished.

So the first few frames of the example can have no msaa, but afterwards it will always be enabled.
I checked it with some printing, and effectively I can have up to ~10 frame without the pipeline ready, then it is always there.