plugins: postgresql add support for PostgreSQL 17 + improvements#2004
Conversation
c967c29 to
6099c78
Compare
6099c78 to
eaf6af1
Compare
8131e61 to
cc58878
Compare
cc58878 to
2588105
Compare
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
systemtests/tests/py3plug-fd-postgresql/database/setup_local_db.sh
Outdated
Show resolved
Hide resolved
docs/adr/XXXX-always-use-integer-when-comparing-numbers-in-python.md
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
c040355 to
f6a6b09
Compare
docs/adr/XXXX-always-use-integer-when-comparing-numbers-in-python.md
Outdated
Show resolved
Hide resolved
docs/adr/XXXX-always-use-integer-when-comparing-numbers-in-python.md
Outdated
Show resolved
Hide resolved
core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py
Outdated
Show resolved
Hide resolved
2f3cbd1 to
a248741
Compare
|
Hi Sebastian, this morning with the formal acceptance of the new ADR, I've renamed the file and squash the commit for a final review. |
There was a problem hiding this comment.
There seems to be a sporadic test error with this PR:
See e.g. here: https://jenkins.bareos.com/blue/organizations/jenkins/bareos/detail/PR-2004/27/tests
It did not backup or restore recovery.signal correctly.
As we've seen some case where Jenkins didn't got all the files, we know it is temporary failing. If recovery.signal was missing the Restore job wouldn't be ok. |
Our plugin is able to work with PostgreSQL 10. Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
- add support for PostgreSQL version 17, we now use dedicated
function pg_walfile_name_offset to retrieve filename and
check if offset is 0.
- database_local: set checkpoint timeout lower.
- add support for nanosecond integer resolution.
+ use ns in self.last_backup_stop_time as datetime object
are not serializable.
+ use ns for start_backup_time and remove datetime usage
for file comparison.
+ this support allow the plugin to not backup each time the
latest wal file.
- add fix for pg8000 < 1.30 returning a string instead tuple
with pg_walfile_name_offset.
- add a check if ROP last_time_backup is large enough
or convert it to ns (ease plugin version migration).
- remove unused modules datetime dateutil.
- use constant for `NANOSECONDS_PER_SECOND` and
`LAST_BACKUP_TIME_WITH_SECONDS`
- limit if/else block by using global functions current_time_ns()
- add comment about time.time_ns() when to remove it (python 3.7 eol)
- use directly os.stat().st_mtime_ns it has been introduced in
python 3.3 and we don't support python <= 3.6.
- add comment about checkpoint_timeout parameter in setup-local-db.
- use stat_obj instead of stat to avoid keyword redefining.
- improve docstring for __decode_lsn_filename_offset which return
True.
- remove unused lsn parameter from __wait_for_wal_archiving.
- add debug message when LAST_BACKUP_TIME_WITH_SECONDS is hit.
Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
Co-authored-by: Sebastian Sura <124262655+sebsura@users.noreply.github.com>
Introduce newly accepted adr for python code: 0003-always-use-integer-when-comparing-timestamp-numbers-in-python Fix typo in README.md Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com> Co-authored-by: Sebastian Sura <124262655+sebsura@users.noreply.github.com>
912841e to
73dd9b8
Compare
This PR aims to fix issue #1982 problem with incremental on PG17
pg_walfile_name_offset to retrieve filename and check if offset is 0.
datetime,dateutilPlease check
If you have any questions or problems, please give a comment in the PR.
Helpful documentation and best practices
Checklist for the reviewer of the PR (will be processed by the Bareos team)
Make sure you check/merge the PR using
devtools/pr-toolto have some simple automated checks run and a proper changelog record added.General
Source code quality