fix: disable management of end-of-wal file flag during backup restoration#8873
Merged
mnencia merged 1 commit intocloudnative-pg:mainfrom Oct 17, 2025
Merged
Conversation
…tion When the end of the WAL stream is reached, the parallel WAL restore feature attempts to predict the names of subsequent WAL files to restore and records the first missing WAL file. On high-availability (HA) replicas, if PostgreSQL requests the first missing WAL file, the code returns an error status that prompts PostgreSQL to switch to streaming replication. Currently, the code assumes a wal_segment_size of 16MB for predicting the next WAL file names. If the configured WAL segment size exceeds 16MB, it may request non-existent WAL files. For instance, with 16MB segments, the names would range from 000000010000000100000000 to 0000000100000001000000FF before moving to the next segment. For 1GB segments, they would range from 000000010000000100000000 to 000000010000000100000003. With the assumption of a 16MB segment size, the code will not find the WALs from 000000010000000100000004 to 0000000100000001000000FF. While this assumption does not affect HA replicas—which can shift to streaming mode—it's problematic for a PostgreSQL instance seeking consistency after a restore, as the restore process will fail. This patch disables end-of-wal file marker management during replication, addressing restore issues for backups that were: 1. using a custom WAL file segment size 2. utilizing parallel WAL recovery 3. initiated on one WAL segment and concluded on a different one Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com>
Contributor
|
❗ By default, the pull request is configured to backport to all release branches.
|
armru
approved these changes
Oct 16, 2025
gbartolini
approved these changes
Oct 16, 2025
Contributor
|
/test |
Contributor
|
@gbartolini, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/18564767910 |
mnencia
approved these changes
Oct 17, 2025
cnpg-bot
pushed a commit
that referenced
this pull request
Oct 17, 2025
…tion (#8873) When the end of the WAL stream is reached, the parallel WAL restore feature attempts to predict the names of subsequent WAL files to restore and records the first missing WAL file. On high-availability (HA) replicas, if PostgreSQL requests the first missing WAL file, the code returns an error status that prompts PostgreSQL to switch to streaming replication. Currently, the code assumes a wal_segment_size of 16MB for predicting the next WAL file names. If the configured WAL segment size exceeds 16MB, it may request non-existent WAL files. For instance, with 16MB segments, the names would range from `000000010000000100000000` to `0000000100000001000000FF` before moving to the next segment. For 1GB segments, they would range from `000000010000000100000000` to `000000010000000100000003`. With the assumption of a 16MB segment size, the code will not find the WALs from `000000010000000100000004` to `0000000100000001000000FF`. While this assumption does not affect HA replicas - which can shift to streaming mode - it's problematic for a PostgreSQL instance seeking consistency after a restore, as the restore process will fail. This patch disables end-of-wal file marker management during replication, addressing restore issues for backups that were: 1. using a custom WAL file segment size 2. utilizing parallel WAL recovery 3. initiated on one WAL segment and concluded on a different one Closes: #8874 Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com> (cherry picked from commit 22f9338)
cnpg-bot
pushed a commit
that referenced
this pull request
Oct 17, 2025
…tion (#8873) When the end of the WAL stream is reached, the parallel WAL restore feature attempts to predict the names of subsequent WAL files to restore and records the first missing WAL file. On high-availability (HA) replicas, if PostgreSQL requests the first missing WAL file, the code returns an error status that prompts PostgreSQL to switch to streaming replication. Currently, the code assumes a wal_segment_size of 16MB for predicting the next WAL file names. If the configured WAL segment size exceeds 16MB, it may request non-existent WAL files. For instance, with 16MB segments, the names would range from `000000010000000100000000` to `0000000100000001000000FF` before moving to the next segment. For 1GB segments, they would range from `000000010000000100000000` to `000000010000000100000003`. With the assumption of a 16MB segment size, the code will not find the WALs from `000000010000000100000004` to `0000000100000001000000FF`. While this assumption does not affect HA replicas - which can shift to streaming mode - it's problematic for a PostgreSQL instance seeking consistency after a restore, as the restore process will fail. This patch disables end-of-wal file marker management during replication, addressing restore issues for backups that were: 1. using a custom WAL file segment size 2. utilizing parallel WAL recovery 3. initiated on one WAL segment and concluded on a different one Closes: #8874 Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com> (cherry picked from commit 22f9338)
cnpg-bot
pushed a commit
that referenced
this pull request
Oct 17, 2025
…tion (#8873) When the end of the WAL stream is reached, the parallel WAL restore feature attempts to predict the names of subsequent WAL files to restore and records the first missing WAL file. On high-availability (HA) replicas, if PostgreSQL requests the first missing WAL file, the code returns an error status that prompts PostgreSQL to switch to streaming replication. Currently, the code assumes a wal_segment_size of 16MB for predicting the next WAL file names. If the configured WAL segment size exceeds 16MB, it may request non-existent WAL files. For instance, with 16MB segments, the names would range from `000000010000000100000000` to `0000000100000001000000FF` before moving to the next segment. For 1GB segments, they would range from `000000010000000100000000` to `000000010000000100000003`. With the assumption of a 16MB segment size, the code will not find the WALs from `000000010000000100000004` to `0000000100000001000000FF`. While this assumption does not affect HA replicas - which can shift to streaming mode - it's problematic for a PostgreSQL instance seeking consistency after a restore, as the restore process will fail. This patch disables end-of-wal file marker management during replication, addressing restore issues for backups that were: 1. using a custom WAL file segment size 2. utilizing parallel WAL recovery 3. initiated on one WAL segment and concluded on a different one Closes: #8874 Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com> (cherry picked from commit 22f9338)
THE-BRAHMA
pushed a commit
to THE-BRAHMA/cloudnative-pg
that referenced
this pull request
Oct 30, 2025
…tion (cloudnative-pg#8873) When the end of the WAL stream is reached, the parallel WAL restore feature attempts to predict the names of subsequent WAL files to restore and records the first missing WAL file. On high-availability (HA) replicas, if PostgreSQL requests the first missing WAL file, the code returns an error status that prompts PostgreSQL to switch to streaming replication. Currently, the code assumes a wal_segment_size of 16MB for predicting the next WAL file names. If the configured WAL segment size exceeds 16MB, it may request non-existent WAL files. For instance, with 16MB segments, the names would range from `000000010000000100000000` to `0000000100000001000000FF` before moving to the next segment. For 1GB segments, they would range from `000000010000000100000000` to `000000010000000100000003`. With the assumption of a 16MB segment size, the code will not find the WALs from `000000010000000100000004` to `0000000100000001000000FF`. While this assumption does not affect HA replicas - which can shift to streaming mode - it's problematic for a PostgreSQL instance seeking consistency after a restore, as the restore process will fail. This patch disables end-of-wal file marker management during replication, addressing restore issues for backups that were: 1. using a custom WAL file segment size 2. utilizing parallel WAL recovery 3. initiated on one WAL segment and concluded on a different one Closes: cloudnative-pg#8874 Signed-off-by: Leonardo Cecchi <leonardo.cecchi@enterprisedb.com> Signed-off-by: theBrahma <office.utpal.brahma@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the end of the WAL stream is reached, the parallel WAL restore feature attempts to predict the names of subsequent WAL files to restore and records the first missing WAL file.
On high-availability (HA) replicas, if PostgreSQL requests the first missing WAL file, the code returns an error status that prompts PostgreSQL to switch to streaming replication.
Currently, the code assumes a wal_segment_size of 16MB for predicting the next WAL file names. If the configured WAL segment size exceeds 16MB, it may request non-existent WAL files. For instance, with 16MB segments, the names would range from
000000010000000100000000to0000000100000001000000FFbefore moving to the next segment. For 1GB segments, they would range from000000010000000100000000to000000010000000100000003.With the assumption of a 16MB segment size, the code will not find the WALs from
000000010000000100000004to0000000100000001000000FF.While this assumption does not affect HA replicas - which can shift to streaming mode - it's problematic for a PostgreSQL instance seeking consistency after a restore, as the restore process will fail.
This patch disables end-of-wal file marker management during replication, addressing restore issues for backups that were:
Closes: #8874