-
Notifications
You must be signed in to change notification settings - Fork 632
Description
Is there an existing issue already for this feature request/idea?
- I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.
What problem is this feature going to solve? Why should it be added?
Importing a db with pg_dump using a single worker is slow. There is also no easy hook for the SELECT timescaledb_pre_restore(); and SELECT timescaledb_post_restore(); functions.
Describe the solution you'd like
A common pattern is to split the job into pgdumps predata, data, and postdata stages. You generally want to use a single worker when copying the schema, many workers when copying the actual data, and at least one worker when applying constraints at the end depending on how fast that is. The current extra settings flag does not give you the control to do this.
I'm willing to work on this feature. Looking at the existing codebase, this does look like a good first issue assuming prior experience with pg_dump, since the import section is fairly self-contained.
Describe alternatives you've considered
Running a shell script by piping to kubectl exec sh). This is pretty close to what we do now, since pg_dump/pg_restore is fairly independent of how it runs.
Other options would be to have a CNPG-I plugin for other restore options like pgcopydb. Another is to not bother with any optimizations but to add six lines to detect whether timescaledb is a preloaded extension and if so to run the SELECT timescaledb_pre_restore(); and SELECT timescaledb_post_restore(); functions which are required to load the backup correctly, with no API changes and only a fix to import databases with a common extension.
Additional context
Exposing a good API for this is tricky and this is something I'd like a maintainer opinion on.
Backport?
No
Are you willing to actively contribute to this feature?
Yes
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
Type
Projects
Status