Skip to content

Refactor GraphLinksConverter#5491

Merged
xzfc merged 3 commits intodevfrom
links-comp-refactor-GraphLinksConverter
Nov 27, 2024
Merged

Refactor GraphLinksConverter#5491
xzfc merged 3 commits intodevfrom
links-comp-refactor-GraphLinksConverter

Conversation

@xzfc
Copy link
Member

@xzfc xzfc commented Nov 20, 2024

This PR refactors GraphLinksConverter in preparation for adding support for compressing.

  • GraphLinksFileHeader is split into GraphLinksFileHeader (on-disk representation) and GraphLinksFileInfo (in-memory, all offsets are calculated).
  • Replaced usages of mmap_ops::transmute_from_u8_to_slice with equivalents from the transmute crate (as I find it to be more sound).
  • Now, it serializes into impl Write sequentially rather than into &mut [u8].
  • The serialization code now looks much simpler. It will extended later to support writing in a compressed format.

@xzfc xzfc force-pushed the links-comp-refactor-GraphLinksConverter branch from 0ae071e to ddd2984 Compare November 26, 2024 21:50
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, though it's a bit hard to track where things have moved exactly.

Thanks for replacing those transmutes.

@xzfc xzfc merged commit ec7caae into dev Nov 27, 2024
@xzfc xzfc deleted the links-comp-refactor-GraphLinksConverter branch November 27, 2024 21:37
coszio pushed a commit that referenced this pull request Dec 2, 2024
* Introduce HEADER_SIZE = 64

* GraphLinksConverter: rewrite

* Split GraphLinksFileHeader into GraphLinksFileInfo
timvisee pushed a commit that referenced this pull request Dec 9, 2024
* Introduce HEADER_SIZE = 64

* GraphLinksConverter: rewrite

* Split GraphLinksFileHeader into GraphLinksFileInfo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants