Skip to content

Codegen'd deserialization refactoring: correctness, error handling, helpers#2953

Merged
teh-cmc merged 21 commits intomainfrom
cmc/deser_refactor
Aug 11, 2023
Merged

Codegen'd deserialization refactoring: correctness, error handling, helpers#2953
teh-cmc merged 21 commits intomainfrom
cmc/deser_refactor

Conversation

@teh-cmc
Copy link
Copy Markdown
Contributor

@teh-cmc teh-cmc commented Aug 9, 2023

Commit by commit

This actually ended up looking more like a massive clean up more than a refactoring per-se, as I've found out after experimenting with a couple alternative designs that what we currently have is actually pretty nice in many aspects.
It just needed a lot of cleaning up is all!

TL;DR:

  • Get rid of remaining adhoc code here and there (e.g. handmade validity iterators etc)
  • Improve error handling APIs
  • Defend against all possible malformed inputs: the viewer shall never crash
  • Abstract the dark art of iterator unmappings
  • Split up Rust codegen into submodules

Specifically this fixes:


By side-effect, this makes transforms work with the C++ SDK:
image
image


Benchmarks vs main just in case:
image

What

Checklist

@teh-cmc teh-cmc added sdk-python Python logging API sdk-rust Rust logging API codegen/idl labels Aug 9, 2023
@teh-cmc teh-cmc marked this pull request as draft August 9, 2023 19:45
@teh-cmc teh-cmc force-pushed the cmc/deser_refactor branch from edecb6a to 0996869 Compare August 10, 2023 07:35
@teh-cmc teh-cmc marked this pull request as ready for review August 10, 2023 08:54
@jleibs jleibs self-requested a review August 10, 2023 15:08
Copy link
Copy Markdown
Contributor

@jleibs jleibs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is such a huge improvement in legibility and error handling. Nice work!

@teh-cmc teh-cmc force-pushed the cmc/deser_refactor branch from c6ac602 to a97a4b8 Compare August 11, 2023 08:34
@teh-cmc teh-cmc force-pushed the cmc/deser_refactor branch from 7d60daf to eb45e2c Compare August 11, 2023 08:38
@teh-cmc
Copy link
Copy Markdown
Contributor Author

teh-cmc commented Aug 11, 2023

Integrated your suggestions, looks great!

@teh-cmc teh-cmc merged commit 3ecc8ef into main Aug 11, 2023
@teh-cmc teh-cmc deleted the cmc/deser_refactor branch August 11, 2023 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codegen/idl sdk-python Python logging API sdk-rust Rust logging API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sanitize error handling on deserialization path Support deserializing illegally nullable data

2 participants