Skip to content

[Feature]: Allow extra pg_dump/pg_restore settings to apply only to predata/postdata #6874

@saolof

Description

@saolof

Is there an existing issue already for this feature request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

Importing a db with pg_dump using a single worker is slow. There is also no easy hook for the SELECT timescaledb_pre_restore(); and SELECT timescaledb_post_restore(); functions.

Describe the solution you'd like

A common pattern is to split the job into pgdumps predata, data, and postdata stages. You generally want to use a single worker when copying the schema, many workers when copying the actual data, and at least one worker when applying constraints at the end depending on how fast that is. The current extra settings flag does not give you the control to do this.

I'm willing to work on this feature. Looking at the existing codebase, this does look like a good first issue assuming prior experience with pg_dump, since the import section is fairly self-contained.

Describe alternatives you've considered

Running a shell script by piping to kubectl exec sh). This is pretty close to what we do now, since pg_dump/pg_restore is fairly independent of how it runs.

Other options would be to have a CNPG-I plugin for other restore options like pgcopydb. Another is to not bother with any optimizations but to add six lines to detect whether timescaledb is a preloaded extension and if so to run the SELECT timescaledb_pre_restore(); and SELECT timescaledb_post_restore(); functions which are required to load the backup correctly, with no API changes and only a fix to import databases with a common extension.

Additional context

Exposing a good API for this is tricky and this is something I'd like a maintainer opinion on.

Backport?

No

Are you willing to actively contribute to this feature?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions