Skip to content

postgresql plugin: read config parameters from cluster#1599

Merged
BareosBot merged 5 commits intobareos:masterfrom
bruno-at-bareos:dev/bruno/master/plugin-postgresql-parameter-from-cluster
Nov 20, 2023
Merged

postgresql plugin: read config parameters from cluster#1599
BareosBot merged 5 commits intobareos:masterfrom
bruno-at-bareos:dev/bruno/master/plugin-postgresql-parameter-from-cluster

Conversation

@bruno-at-bareos
Copy link
Contributor

@bruno-at-bareos bruno-at-bareos commented Nov 15, 2023 •
edited by pstorz
Loading

With the previous version of the code, the plugin during restore could overwrite real data if the given data_dir was a symlink.
This new version connects to the cluster and asks directly its configuration parameters.

  • remove no more used parameter postgresql_data_dir
  • wal_archive_dir parameter is checked if it is a symlink, if the case the real location is retrieved and used instead of the symlink, a warning is added to the joblog (testrunner-walsymlink)
  • backup external configuration: The plugin now tries its best to backup the directory content where postgresql.conf resides. This is important especially on on Debian-based OS, as there the configuration is unfortunately not stored in the data directory but in /etc/postgesql/<version>/<instance>/ when using the pg_cluster tools.
  • add a systemtest to test debian/ubuntu specific location and tools.
  • private function will use python raise and they are called inside try: except: block in main function for a better error handling and stopping failed jobs as early as possible.
  • documentation got small changes (removal of parameters) and added warnings (do a full after restore)

Fix Issue #1563
OP#5610

Please check

  • Short description and the purpose of this PR is present above this paragraph
  • Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General
  • Is the PR title usable as CHANGELOG entry?
  • Purpose of the PR is understood
  • Commit descriptions are understandable and well formatted
  • Check backport line
  • Required backport PRs have been created
Source code quality
  • Source code changes are understandable
  • Variable and function names are meaningful
  • Code comments are correct (logically and spelling)
  • Required documentation changes are present and part of the PR
Tests
  • Decision taken that a test is required (if not, then remove this paragraph)
  • The choice of the type of test (unit test or systemtest) is reasonable
  • Testname matches exactly what is being tested
  • On a fail, output of the test leads quickly to the origin of the fault

@bruno-at-bareos bruno-at-bareos changed the title Dev/bruno/master/plugin postgresql parameter from cluster plugin postgresql parameter from cluster Nov 15, 2023
@bruno-at-bareos bruno-at-bareos force-pushed the dev/bruno/master/plugin-postgresql-parameter-from-cluster branch from ad54a88 to dc4c0b1 Compare November 16, 2023 15:08
pstorz
pstorz requested changes Nov 16, 2023
View reviewed changes
Copy link
Member

@pstorz pstorz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments. Good work!

@bruno-at-bareos bruno-at-bareos marked this pull request as ready for review November 20, 2023 08:44
bruno-at-bareos and others added 3 commits November 20, 2023 09:59
@bruno-at-bareos
PostgreSQL plugin: improvements / fixes
67c6252 
- adjust plugin code to ask configuration parameters from the cluster
- use `raise` error in private functions, and `except` in main
- apply black modify suggestions
- reorder alphabetically the private functions
- delete removed parameter postgresql_data_directory
- add wal_archive_symlink for testing
- add systemtest testrunner-walsymlink for wal_archive_dir as symlink
- add systemtest testrunner-debian for system with pg_*cluster* tools
  add mention about support of debian like pg_*cluster* tools
- testrunner-roles adjust sentence checked, fix start_fast parameter
- setup_database_local.sh fix ID usage and double space

Fix issue bareos#1561

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos
docs: PostgreSQL plugin improvements
a146465 
- remove no more used parameters
- add roles requirements sql examples
- add socket explanation for db_host
- add warning after restore: clean older wals and trigger new full
- fix typos

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
@bruno-at-bareos @pstorz
Apply suggestions from code review
ae05929 
Co-authored-by: Philipp Storz <philipp.storz@bareos.com>
@bruno-at-bareos bruno-at-bareos force-pushed the dev/bruno/master/plugin-postgresql-parameter-from-cluster branch from 1d31c37 to ae05929 Compare November 20, 2023 08:59
@pstorz pstorz changed the title plugin postgresql parameter from cluster postgresql plugin: read config parameters from cluster Nov 20, 2023
@BareosBot BareosBot merged commit daef118 into bareos:master Nov 20, 2023
@bruno-at-bareos bruno-at-bareos deleted the dev/bruno/master/plugin-postgresql-parameter-from-cluster branch January 15, 2024 13:47
sebastianlederer pushed a commit to sebastianlederer/bareos that referenced this pull request Feb 27, 2024
Bugfix: clean up error handling in LDAP plugin
b7e0680 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
sebastianlederer pushed a commit to sebastianlederer/bareos that referenced this pull request Feb 27, 2024
Bugfix: clean up error handling in LDAP plugin
6d7c3a7 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
sebastianlederer pushed a commit to sebastianlederer/bareos that referenced this pull request Feb 27, 2024
Bugfix: clean up error handling in LDAP plugin
430042d 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
joergsteffens pushed a commit to joergsteffens/bareos that referenced this pull request Feb 28, 2024
@joergsteffens
Bugfix: clean up error handling in LDAP plugin
b1609f0 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
(cherry picked from commit 430042d)
sebastianlederer pushed a commit to sebastianlederer/bareos that referenced this pull request Mar 7, 2024
Bugfix: clean up error handling in LDAP plugin
f8f24c8 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
BareosBot pushed a commit to joergsteffens/bareos that referenced this pull request Mar 12, 2024
@BareosBot
Bugfix: clean up error handling in LDAP plugin
fe6ccef 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
(cherry picked from commit 430042d)
banuchka pushed a commit to banuchka/bareos that referenced this pull request Mar 22, 2024
@banuchka
Bugfix: clean up error handling in LDAP plugin
9699d00 
- handle referrals in search results
- better error reporting when using incorrect base DN
- refactor reporting of LDAP exceptions
- replace old-style % operator with format()
- Debian package build: fix dependencies for python-ldap

Fixes: bareos#1599: FD LDAP Plugin has broken dependencies on Debian 11+
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pstorz pstorz pstorz approved these changes

Assignees

@pstorz pstorz

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bruno-at-bareos @pstorz @BareosBot