Switching to serializing dynamic memory with minimum 8 byte alignment instead of 16. This is for a few reasons: - Most data types don't have 16 byte alignment, so we'd be wasting bytes and generating a bigger serialized blob than we'd need to. - There's not a clean way to alloc the result buffer with an initial alignment of 16 (or more) without having to return an extra pointer to the user to free. But most allocators do align to 8 internally. - The thing that requires more than 8 byte alignment most often (at least for my use case) would be data you want to do SIMD aligned loads from, so 128, 256 or 512-bit aligned loads into SIMD registers. But apparently unaligned and aligned loads are equally as fast on current hardware so this shouldn't be that much of a concern. Ideally we'd like to align the dynamic memory elements to the actual alignment their type has, but we don't have an alignment_of(T) or an alignment field in the Type_Info so no luck there. The language in general doesn't expose much in terms of alignment at the time of writing. Also ideally we'd love to have allocators support alignment natively so we don't have to keep two pointers, one for the data and another one only for freeing later. Finally, had to remove some usages of `#must` since it has been unfortunately deprecated. |
||
|---|---|---|
| example.jai | ||
| LICENSE | ||
| module.jai | ||
| README.md | ||
💽 A Serialization Module for Jai
A wee serialization module for the Jai programming language. You can check the code in module.jai then a few examples of how to use them in example.jai.
How To
You can call serialize to transform a data structure into a byte buffer (including stuff like arrays, strings, pointers and potentially other dynamically allocated stuff). This only allocates a single time to get a buffer big enough to fit all the data on your type + any variable size data. It can also use a given allocator, defaulting to the current context.allocator.
data : Data;
bytes := serialize(*data);
store_bytes_somewhere(bytes);
Then you can read back those bytes and transform them to a pointer of the original size by calling deserialize. This casts the bytes to a pointer of the type you serialized and resolves any dynamic data. No allocations happen at this stage, just some data fixup so everything points to where it should in memory.
bytes := read_bytes_from_somewhere();
data := deserialize(bytes, Data);
happily_use_the_data(data);
Notes
Part of the purpose of this module was to explore how to do serialization/deserialization fast using the features that Jai offers that aren't in most languages. It makes extensive use of the type information that the language gives but also makes good use of compile-time code generation. For each type that the code will serialize/deserialize it will generate bespoke code to handle its members. The generated code is nice and fast but it's good to take into account that if this is overused it might make a dent in compile times and potentially code size, although I haven't experienced either in any significant way myself at the time of writing.
License
I want people to be able to just use this without thinking about licensing, although give me a shout if you do use it and attribution is appreciated 😊. Replicating STB's (https://github.com/nothings/stb) licensing stuff where you can choose to use public domain or MIT.