Skip to content

Conversation

@gowsiany
Copy link
Contributor

@gowsiany gowsiany commented May 2, 2025

Fixes #8590

Summary

This PR corrects vtable deduplication for FlatBuffers using 64-bit offsets. Previously, when serializing buffers larger than 2GB, vtable deduplication failed, resulting in much slower serialization and larger buffer sizes.

Root Cause

The issue was caused by 64-bit offset calculations not accounting for length_of_64_bit_region_. As a result, the addresses of vtable candidates stored in scratch_data were miscalculated, preventing proper deduplication.

Solution

This fix includes length_of_64_bit_region_ in the offset calculation for vtables in 64-bit mode. This ensures correct vtable candidate addresses, enabling deduplication and restoring efficient serialization for large buffers.

Impact

  • Restores vtable deduplication for buffers >2GB with 64-bit offsets.
  • Significantly improves serialization performance and reduces buffer size for large data sets.
  • Maintains full backward compatibility with 32-bit buffers.

Verification

  • Serialization benchmarks with large (2GB+) buffers now show linear scaling and reduced buffer size.

@github-actions github-actions bot added the c++ label May 2, 2025
@fergushenderson
Copy link
Contributor

Thanks for the fix.
It would be nice to add a regression test.

@jtdavis777 jtdavis777 self-requested a review November 25, 2025 00:08
Copy link
Collaborator

@jtdavis777 jtdavis777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I didn't fix the formatting properly while fixing a merge conflict, but I think it's fine. a pass of the formatter at a later time will fix it and it doesn't affect readability.

@jtdavis777 jtdavis777 merged commit 8b02fe6 into google:master Nov 27, 2025
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++] Vtable deduplication fails for 64-bit buffers >2GB, causing slow serialization and larger buffers

3 participants