Skip to content

Add new param to retain more closed segments#93

Merged
KShivendu merged 9 commits intomasterfrom
max-closed-to-preserve
Aug 5, 2025
Merged

Add new param to retain more closed segments#93
KShivendu merged 9 commits intomasterfrom
max-closed-to-preserve

Conversation

@KShivendu
Copy link
Member

@KShivendu KShivendu commented Jul 30, 2025

Whenever we are doing prefix_truncate() in the wal, we can preserve more closed segments. The current value is 1 (implicit) and hence kept it as default for backwards compatibility.

The advantage is that it can make it more likely to use wal delta transfer over streaming records which is faster than the latter.

Base automatically changed from clippy-fixes to master July 30, 2025 09:41
@KShivendu KShivendu marked this pull request as draft July 30, 2025 09:42
@KShivendu KShivendu marked this pull request as ready for review July 30, 2025 14:27
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add a test to assert the new behavior works

@KShivendu
Copy link
Member Author

Please also add a test to assert the new behavior works

@timvisee already did the automated property tests that tests against 100 random values. I'm adding a normal test in #94. Will add retain_closed there as well after merging it.

@KShivendu KShivendu requested a review from timvisee August 1, 2025 18:14
@KShivendu KShivendu changed the title Add new param for max closed wal segments to preserve Add new param for max closed wal segments to retain Aug 4, 2025
@KShivendu KShivendu changed the title Add new param for max closed wal segments to retain Add new param for retain more closed segments Aug 4, 2025
@KShivendu KShivendu changed the title Add new param for retain more closed segments Add new param to retain more closed segments Aug 4, 2025
@KShivendu KShivendu force-pushed the max-closed-to-preserve branch from 9b89c83 to be03389 Compare August 4, 2025 18:06
@KShivendu KShivendu changed the base branch from master to test-prefix-truncate August 4, 2025 18:06
@KShivendu KShivendu force-pushed the max-closed-to-preserve branch from 25eb04f to 62431f5 Compare August 4, 2025 19:36
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done quite a bit of testing on this one, all well and good. Thanks!

Base automatically changed from test-prefix-truncate to master August 5, 2025 12:51
@KShivendu KShivendu merged commit fad0593 into master Aug 5, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants