This is the start of my very roundabout way to get back to #444, where we really need a benchmark that captures something other than JSON-like-RON to ron::Value. I hope to upgrade our arbitrary fuzzer to use proper typing to generate an arbitrary data structure and its corresponding Serialize and Deserialize implementation. For a new PR, we would then first run the fuzzer, then extract the corpus for the arbitrary target, and then benchmark serialising and deserialising based on these examples. Ideally, the current main branch would also be pulled in again and run on these benchmarks as well to provide an automatic comparison.

This will probably take me several weekends to fully implement, but I hope it will finally give us the needed insights to land #444 with the best perf-maintainability tradeoff.

• I've included my change in CHANGELOG.md

Add tests to document the following bugs found by fuzzing and now fixed:

• struct, enum, and variant names are always validated
• unit structs / variants called r can be parsed by ron::Value (which previously thought this was the start of a raw string)
• strings containing '\\' are serialised as raw strings when escaping is turned off
• a stack of nested Options which are serialised with #![enable(implicit_some)] and contains a None cannot be uniquely deserialised, since we have no idea where the None came from. This case has to be tracked, so that Somes can be inserted in case a None is detected inside an unbroken stack of implicit Somes.
• deserialising "A('/')" into ron::Value fails as the struct type searcher reads into the char and then finds a weird comment starter there

Problematic bugs which need to be documented, tested, and discussed further:

• deserialising Some(...) inside deserialize_any with #![enable(unwrap_variant_newtypes)] cannot work as currently implemented, thus it is now properly detected with a new (very specific) error code. Unwrapping variant newtypes currently reaches through Options, and [v0.9] Breaking: Treat Some like any newtype variant #413 makes it more explicit by treating Some like a newtype variant. However, deserialize_any cannot support newtype variant Some in all cases, since it special-cases Some(...) to look at .... E.g. Some(a: 4) works great in typed mode and looks very nice, but cannot be supported here. Either we decide to make Some explicitly not a newtype variant (which is a breaking change since it kind of escaped through it before and loses us the nice syntax), or we keep this very obscure error which should not be encountered often. The former would definitely be safer. Another alternative is to use Add minimal support for internally tagged and untagged enums #451 to pre-parse the struct type in deserialize_any when unwrap_variant_newtypes is enabled and to handle tuples, structs, and unit structs with special cases.

Future work

• can we also fuzz over the serde flatten and enum attributes?
• the struct type guesser introduced in Add minimal support for internally tagged and untagged enums #451 is quite expensive and can lead to quadratic parsing times. Perhaps we could cache some information / an AST would also work better here.