application\n |\n VIP 10.0.1.10 (Keepalived: floats between ha1\/ha2)\n |\n +----------+----------+\n | ha1 ha2 | HAProxy (VRRP MASTER \/ BACKUP)\n | :5000 writes -> \/primary (200 only on the leader)\n | :5001 reads -> \/replica (200 only on standbys)\n +----------+----------+\n | health check on :8008\n +----------+-----------+-----------+\n | pg1 pg2 pg3 |\n | Patroni Patroni Patroni |\n | etcd etcd etcd | 3-node DCS quorum\n | PostgreSQL PostgreSQL PostgreSQL|\n +-----------------------------------+\n primary <-- streaming --> standbys<\/code><\/pre>\n\n\n\nThe roles break down like this:<\/p>\n\n\n\n
\n- Patroni<\/strong> runs on every database node. It manages PostgreSQL, watches its health, and races for a leader lock. Whoever holds the lock is the primary. If the lock expires, Patroni promotes a standby and demotes the old primary the moment it comes back.<\/li>\n
- etcd<\/strong> is the distributed configuration store that holds the leader lock. It uses the Raft consensus algorithm, so only one node can hold the lock at a time. This is what makes split-brain impossible at the cluster layer. It needs an odd number of members, three at minimum, for a quorum.<\/li>\n
- HAProxy<\/strong> is the single address clients connect to. It does not guess who the primary is. It asks Patroni’s REST API on each node and routes accordingly, so when the primary changes, the backend changes but the client’s endpoint never does.<\/li>\n
- Keepalived<\/strong> gives HAProxy the same treatment HAProxy gives PostgreSQL. A floating virtual IP moves between two HAProxy nodes using VRRP, so a dead proxy is not a dead cluster.<\/li>\n<\/ul>\n\n\n\n
The trade-off worth naming up front: this is five machines to run one logical database. If you only need a warm standby for disaster recovery and can tolerate a manual promotion, plain streaming replication<\/a> is simpler and cheaper. Reach for the full stack when an unplanned primary loss has to heal itself in under a minute with no human in the loop.<\/p>\n\n\n\nPrerequisites<\/h2>\n\n\n\n
Five nodes give the cleanest separation: three for the database and etcd, two for the proxy layer. You can collapse the proxy onto the database nodes in a lab, but keeping them apart is what you want in production.<\/p>\n\n\n\n
\n- Three database nodes (
pg1<\/code>, pg2<\/code>, pg3<\/code>) and two proxy nodes (ha1<\/code>, ha2<\/code>), each a recent Linux server with at least 2 vCPU and 2 GB RAM.<\/li>\n- Static IPs on the same subnet, plus one free address for the virtual IP. This guide uses
10.0.1.11-13<\/code> for the database nodes, 10.0.1.21-22<\/code> for the proxies, and 10.0.1.10<\/code> for the VIP.<\/li>\n- Tested on: PostgreSQL 18, Patroni 4.1.3, etcd 3.4-3.6, HAProxy 3.0, Keepalived 2.2, on Rocky Linux 10 \/ AlmaLinux 10, Debian 13, and Ubuntu 24.04 \/ 26.04.<\/li>\n
- Time synchronisation (chrony) running on every node. Clock skew breaks lease timing.<\/li>\n<\/ul>\n\n\n\n
If PostgreSQL is new to you, the single-server install guides for Rocky and AlmaLinux<\/a> and for Ubuntu and Debian<\/a> cover the basics this guide assumes.<\/p>\n\n\n\nSet the cluster variables<\/h2>\n\n\n\n
The same addresses and passwords appear in dozens of commands. Export them once at the top of each shell session so you edit one block and paste the rest unchanged. Set these on whichever node you are working on, swapping the real values for yours.<\/p>\n\n\n\n
export PG1=10.0.1.11\nexport PG2=10.0.1.12\nexport PG3=10.0.1.13\nexport VIP=10.0.1.10\nexport SUBNET=10.0.1.0\/24\n# Pick real passwords. These three accounts drive replication and failover.\nexport SUPERUSER_PWD='ChangeMe-Super#2026'\nexport REPL_PWD='ChangeMe-Repl#2026'\nexport REWIND_PWD='ChangeMe-Rewind#2026'<\/code><\/pre>\n\n\n\nThese hold for the current session only. Re-export them if you reconnect or switch to a root shell.<\/p>\n\n\n\n
Install PostgreSQL, Patroni and etcd<\/h2>\n\n\n\n
This is the one step where the distributions genuinely diverge. The package names, the repository setup, and one important gotcha differ between the RHEL family and the Debian family. Everything after this section is identical across all of them. Run the install on all three database nodes.<\/p>\n\n\n\n
On the RHEL family (Rocky Linux, AlmaLinux)<\/h3>\n\n\n\n
Add the PGDG repository and EPEL, which carries some of Patroni’s Python dependencies:<\/p>\n\n\n\n
sudo dnf install -y https:\/\/download.postgresql.org\/pub\/repos\/yum\/reporpms\/EL-10-x86_64\/pgdg-redhat-repo-latest.noarch.rpm\nsudo dnf install -y https:\/\/dl.fedoraproject.org\/pub\/epel\/epel-release-latest-10.noarch.rpm<\/code><\/pre>\n\n\n\netcd and Patroni live in the PGDG “extras” repository, which ships disabled. Enable it, then import its signing key. On Enterprise Linux 10 this is the classic config-manager syntax, not the newer dnf5 form, and the key import is required because the extras repo verifies its own metadata:<\/p>\n\n\n\n
sudo dnf config-manager --set-enabled pgdg-rhel10-extras\nsudo rpm --import \/etc\/pki\/rpm-gpg\/PGDG-RPM-GPG-KEY-RHEL<\/code><\/pre>\n\n\n\nSkip that rpm --import<\/code> and the next command fails with repomd.xml GPG signature verification error: Signing key not found<\/code>. With the key in place, install the stack:<\/p>\n\n\n\nsudo dnf install -y postgresql18-server postgresql18-contrib etcd patroni patroni-etcd<\/code><\/pre>\n\n\n\nNote the patroni-etcd<\/code> package. On the RHEL family it pulls the etcd driver Patroni needs. Do not run initdb<\/code> or enable the postgresql<\/code> service here. Patroni initialises the data directory itself.<\/p>\n\n\n\nOn Debian and Ubuntu<\/h3>\n\n\n\n
Add the PGDG apt repository keyed to your release codename (the variable resolves to trixie<\/code>, bookworm<\/code>, noble<\/code>, or resolute<\/code> automatically):<\/p>\n\n\n\nsudo apt-get update\nsudo apt-get install -y curl ca-certificates\nsudo install -d \/usr\/share\/postgresql-common\/pgdg\nsudo curl -fsSL -o \/usr\/share\/postgresql-common\/pgdg\/apt.postgresql.org.asc https:\/\/www.postgresql.org\/media\/keys\/ACCC4CF8.asc\n. \/etc\/os-release\necho \"deb [signed-by=\/usr\/share\/postgresql-common\/pgdg\/apt.postgresql.org.asc] https:\/\/apt.postgresql.org\/pub\/repos\/apt ${VERSION_CODENAME}-pgdg main\" | sudo tee \/etc\/apt\/sources.list.d\/pgdg.list\nsudo apt-get update<\/code><\/pre>\n\n\n\nInstall PostgreSQL, Patroni, and etcd. etcd was renamed on recent Debian and Ubuntu releases, so the package is etcd-server<\/code> and etcd-client<\/code>, not the old etcd<\/code>:<\/p>\n\n\n\nsudo apt-get install -y postgresql-18 patroni etcd-server etcd-client python3-etcd<\/code><\/pre>\n\n\n\nThat python3-etcd<\/code> package is easy to miss and Patroni will not tell you politely. The Debian Patroni package does not pull the etcd driver the way the RHEL patroni-etcd<\/code> package does, so without it Patroni starts and immediately dies with Can not find suitable configuration of distributed configuration store. Available implementations: consul, kubernetes<\/code>. Installing python3-etcd<\/code> puts etcd back on that list.<\/p>\n\n\n\nNow the gotcha that catches everyone. Installing postgresql-18<\/code> on Debian or Ubuntu automatically creates and starts a cluster called main<\/code> on port 5432. Patroni needs that port free and an empty data directory, so drop the auto-created cluster on every node:<\/p>\n\n\n\npg_lsclusters\nsudo pg_dropcluster --stop 18 main<\/code><\/pre>\n\n\n\nThe RHEL packages have no equivalent step, because they never auto-initialise a cluster. This single difference is the most common reason a Debian Patroni node refuses to bootstrap.<\/p>\n\n\n\n
What actually differs between the families<\/h3>\n\n\n\n
Keep this table handy. Every path and name below shows up again in the config files, and getting one wrong is the difference between a clean start and a cryptic Python traceback.<\/p>\n\n\n\nItem<\/th> RHEL family (Rocky, Alma)<\/th> Debian, Ubuntu<\/th><\/tr><\/thead> PostgreSQL package<\/td> postgresql18-server<\/code><\/td>postgresql-18<\/code><\/td><\/tr>\netcd driver for Patroni<\/td> patroni-etcd<\/code><\/td>python3-etcd<\/code><\/td><\/tr>\netcd package<\/td> etcd<\/code><\/td>etcd-server<\/code> etcd-client<\/code><\/td><\/tr>\nAuto-created cluster<\/td> none<\/td> drop with pg_dropcluster --stop 18 main<\/code><\/td><\/tr>\nPatroni binaries<\/td> \/usr\/pgsql-18\/bin<\/code><\/td>\/usr\/lib\/postgresql\/18\/bin<\/code><\/td><\/tr>\nData directory<\/td> \/var\/lib\/pgsql\/18\/data<\/code><\/td>\/var\/lib\/postgresql\/18\/main<\/code><\/td><\/tr>\netcd config file<\/td> \/etc\/etcd\/etcd.conf<\/code><\/td>\/etc\/default\/etcd<\/code><\/td><\/tr>\nPatroni config file<\/td> \/etc\/patroni\/patroni.yml<\/code><\/td>\/etc\/patroni\/config.yml<\/code><\/td><\/tr>\nFirewall<\/td> firewalld<\/td> ufw<\/td><\/tr>\n Mandatory access control<\/td> SELinux (enforcing)<\/td> AppArmor (no action needed)<\/td><\/tr>\n<\/tbody><\/table><\/figure>\n\n\n\nOpen the firewall ports<\/h2>\n\n\n\n
The cluster speaks on a handful of ports: PostgreSQL on 5432, the Patroni REST API on 8008, and etcd on 2379 (clients) and 2380 (peers). Open these on all three database nodes.<\/p>\n\n\n\n
On the RHEL family, firewalld is the tool:<\/p>\n\n\n\n
sudo firewall-cmd --permanent --add-port={5432,8008,2379,2380}\/tcp\nsudo firewall-cmd --reload<\/code><\/pre>\n\n\n\nOn Debian and Ubuntu, the same ports through ufw:<\/p>\n\n\n\n
sudo ufw allow proto tcp to any port 5432,8008,2379,2380<\/code><\/pre>\n\n\n\nThe two proxy nodes need 5000, 5001, and 7000 open for the HAProxy front ends and stats page, plus the VRRP protocol for Keepalived. We will cover those when we set up the proxy layer.<\/p>\n\n\n\n
Bootstrap the etcd quorum<\/h2>\n\n\n\n
etcd comes first, because Patroni has nowhere to store the leader lock without it. Configure all three members as a static cluster. On the RHEL family the config lives in \/etc\/etcd\/etcd.conf<\/code>; on Debian and Ubuntu it is \/etc\/default\/etcd<\/code>. The contents are the same environment variables either way. Here is pg1<\/code> (set the matching self-address on each node):<\/p>\n\n\n\nETCD_NAME=pg1\nETCD_DATA_DIR=\/var\/lib\/etcd\nETCD_LISTEN_PEER_URLS=http:\/\/10.0.1.11:2380\nETCD_LISTEN_CLIENT_URLS=http:\/\/10.0.1.11:2379,http:\/\/127.0.0.1:2379\nETCD_INITIAL_ADVERTISE_PEER_URLS=http:\/\/10.0.1.11:2380\nETCD_ADVERTISE_CLIENT_URLS=http:\/\/10.0.1.11:2379\nETCD_INITIAL_CLUSTER=pg1=http:\/\/10.0.1.11:2380,pg2=http:\/\/10.0.1.12:2380,pg3=http:\/\/10.0.1.13:2380\nETCD_INITIAL_CLUSTER_STATE=new\nETCD_INITIAL_CLUSTER_TOKEN=pg-etcd<\/code><\/pre>\n\n\n\nRepeat on pg2<\/code> and pg3<\/code>, changing ETCD_NAME<\/code> and the three self-addresses. The ETCD_INITIAL_CLUSTER<\/code> line stays identical on all three. Start etcd on every node within a few seconds of each other so the initial election succeeds:<\/p>\n\n\n\nsudo systemctl enable --now etcd<\/code><\/pre>\n\n\n\nConfirm all three members joined and every endpoint is healthy:<\/p>\n\n\n\n
etcdctl --endpoints=http:\/\/${PG1}:2379,http:\/\/${PG2}:2379,http:\/\/${PG3}:2379 endpoint health -w table<\/code><\/pre>\n\n\n\nAll three should report true<\/code> in the HEALTH column:<\/p>\n\n\n\n+-----------------------+--------+------------+-------+\n| ENDPOINT | HEALTH | TOOK | ERROR |\n+-----------------------+--------+------------+-------+\n| http:\/\/10.0.1.11:2379 | true | 2.21561ms | |\n| http:\/\/10.0.1.12:2379 | true | 2.32060ms | |\n| http:\/\/10.0.1.13:2379 | true | 1.62423ms | |\n+-----------------------+--------+------------+-------+<\/code><\/pre>\n\n\n\nThree is the minimum for a real quorum. With three members the cluster survives one failure and keeps a majority; drop to two and a single loss takes the whole DCS offline, which would freeze every failover decision.<\/p>\n\n\n\n
Configure Patroni<\/h2>\n\n\n\n
Patroni reads one YAML file per node. The only fields that change between nodes are name<\/code> and the two connect_address<\/code> lines. Write this to \/etc\/patroni\/patroni.yml<\/code> on the RHEL family, or \/etc\/patroni\/config.yml<\/code> on Debian and Ubuntu. The example below is the RHEL family version for pg1<\/code>; the inline comment marks the one block Debian and Ubuntu change.<\/p>\n\n\n\nscope: pg-cluster\nnamespace: \/service\/\nname: pg1\n\nrestapi:\n listen: 0.0.0.0:8008\n connect_address: 10.0.1.11:8008\n\netcd3:\n hosts:\n - 10.0.1.11:2379\n - 10.0.1.12:2379\n - 10.0.1.13:2379\n\nbootstrap:\n dcs:\n ttl: 30\n loop_wait: 10\n retry_timeout: 10\n maximum_lag_on_failover: 1048576\n synchronous_mode: true\n postgresql:\n use_pg_rewind: true\n use_slots: true\n parameters:\n wal_level: replica\n hot_standby: \"on\"\n max_wal_senders: 10\n max_replication_slots: 10\n wal_keep_size: 256MB\n wal_log_hints: \"on\"\n initdb:\n - encoding: UTF8\n - data-checksums\n pg_hba:\n - host all all 127.0.0.1\/32 scram-sha-256\n - host all all 10.0.1.0\/24 scram-sha-256\n - host replication replicator 10.0.1.0\/24 scram-sha-256\n\npostgresql:\n listen: 0.0.0.0:5432\n connect_address: 10.0.1.11:5432\n data_dir: \/var\/lib\/pgsql\/18\/data # Debian\/Ubuntu: \/var\/lib\/postgresql\/18\/main\n bin_dir: \/usr\/pgsql-18\/bin # Debian\/Ubuntu: \/usr\/lib\/postgresql\/18\/bin\n authentication:\n superuser:\n username: postgres\n password: 'ChangeMe-Super#2026'\n replication:\n username: replicator\n password: 'ChangeMe-Repl#2026'\n rewind:\n username: rewind_user\n password: 'ChangeMe-Rewind#2026'\n parameters:\n password_encryption: scram-sha-256\n\nwatchdog:\n mode: automatic\n device: \/dev\/watchdog\n safety_margin: 5\n\ntags:\n nofailover: false\n noloadbalance: false\n clonefrom: false\n nosync: false<\/code><\/pre>\n\n\n\nTwo settings carry most of the weight here. synchronous_mode: true<\/code> tells Patroni to keep one standby in lockstep with the primary and only ever promote that standby, which is what gives you a zero data loss failover. use_pg_rewind: true<\/code> lets a failed primary rejoin as a standby without a full re-clone once it recovers. The watchdog<\/code> block is the last line of defence against split-brain, covered in the next section.<\/p>\n\n\n\nSet the file ownership so the postgres<\/code> user can read it, then move on to the watchdog before starting anything:<\/p>\n\n\n\nsudo chown postgres:postgres \/etc\/patroni\/patroni.yml\nsudo chmod 600 \/etc\/patroni\/patroni.yml<\/code><\/pre>\n\n\n\nWire up the watchdog<\/h2>\n\n\n\n
etcd’s lock stops two nodes from both believing they are primary at the cluster layer, but there is a nastier edge case: a primary that is hung or paused long enough to lose its lock, then wakes up and keeps accepting writes for a few seconds before it notices. A hardware or software watchdog closes that window by resetting the node if Patroni stops feeding it a heartbeat. The Linux softdog<\/code> module provides one on any machine, virtual or physical.<\/p>\n\n\n\nLoad it at boot and let Patroni hand ownership of the device to the postgres<\/code> user. Run this on all three database nodes:<\/p>\n\n\n\necho softdog | sudo tee \/etc\/modules-load.d\/softdog.conf\nsudo modprobe softdog\nsudo install -d \/etc\/systemd\/system\/patroni.service.d\nprintf '[Service]\\nExecStartPre=+\/sbin\/modprobe softdog\\nExecStartPre=+\/bin\/chown postgres \/dev\/watchdog\\n' | sudo tee \/etc\/systemd\/system\/patroni.service.d\/watchdog.conf\nsudo systemctl daemon-reload<\/code><\/pre>\n\n\n\nWith mode: automatic<\/code> in the config, Patroni uses the watchdog when it is present and carries on without it when it is not, which is the right default for mixed environments. For maximum safety in production, switch that to mode: required<\/code> so a node refuses to become leader if it cannot arm the watchdog. The trade-off is real: required<\/code> means a node with a misconfigured watchdog will sit out rather than serve, so test it before you rely on it.<\/p>\n\n\n\nStart the cluster<\/h2>\n\n\n\n
Start pg1<\/code> first and let it become the leader. It runs initdb<\/code>, creates the three roles, and takes the lock:<\/p>\n\n\n\nsudo systemctl enable --now patroni<\/code><\/pre>\n\n\n\nGive it half a minute, then start Patroni on pg2<\/code> and pg3<\/code>. They clone from the leader with pg_basebackup<\/code> and come up as standbys. Check the cluster from any node (use config.yml<\/code> in place of patroni.yml<\/code> on Debian and Ubuntu):<\/p>\n\n\n\nsudo patronictl -c \/etc\/patroni\/patroni.yml list<\/code><\/pre>\n\n\n\nOne leader, one synchronous standby, and a streaming replica, all on the same timeline with zero lag:<\/p>\n\n\n\n![\"patronictl](\"https:\/\/computingforgeeks.com\/wp-content\/uploads\/2026\/06\/wm-postgresql-ha-patroni-cluster-status.png\" "\\"\\"")
<\/figure>\n\n\n\nThe Sync Standby<\/code> role is the visible proof that synchronous_mode<\/code> is working. That node has every committed transaction the primary has, which is exactly why Patroni will only ever promote it.<\/p>\n\n\n\nPut HAProxy in front<\/h2>\n\n\n\n
HAProxy is what turns three database nodes into one address. The clever part is the health check: rather than guess which node is primary, HAProxy calls Patroni’s REST API on port 8008. Patroni answers 200<\/code> on \/primary<\/code> only on the leader and 503<\/code> everywhere else, and the mirror image on \/replica<\/code>. So a front end that checks \/primary<\/code> naturally pools only the leader, and one that checks \/replica<\/code> pools only the standbys. When the primary changes, the checks flip and HAProxy follows within a couple of seconds.<\/p>\n\n\n\nInstall HAProxy on both proxy nodes:<\/p>\n\n\n\n
sudo dnf install -y haproxy # RHEL family\nsudo apt-get install -y haproxy # Debian, Ubuntu<\/code><\/pre>\n\n\n\nWrite the same \/etc\/haproxy\/haproxy.cfg<\/code> to both nodes. Port 5000 carries writes to the primary, 5001 load-balances reads across the standbys, and 7000 serves the stats page:<\/p>\n\n\n\nglobal\n maxconn 2000\n log \/dev\/log local0\n\ndefaults\n log global\n mode tcp\n retries 2\n timeout client 30m\n timeout server 30m\n timeout connect 4s\n timeout check 5s\n\nlisten stats\n mode http\n bind *:7000\n stats enable\n stats uri \/\n stats refresh 5s\n\n# Writes: only the Patroni leader answers 200 on \/primary\nlisten postgres_primary\n bind *:5000\n option httpchk\n http-check send meth OPTIONS uri \/primary\n http-check expect status 200\n default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions\n server pg1 10.0.1.11:5432 maxconn 200 check port 8008\n server pg2 10.0.1.12:5432 maxconn 200 check port 8008\n server pg3 10.0.1.13:5432 maxconn 200 check port 8008\n\n# Reads: every healthy standby answers 200 on \/replica\nlisten postgres_replicas\n bind *:5001\n option httpchk\n http-check send meth OPTIONS uri \/replica\n http-check expect status 200\n balance roundrobin\n default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions\n server pg1 10.0.1.11:5432 maxconn 200 check port 8008\n server pg2 10.0.1.12:5432 maxconn 200 check port 8008\n server pg3 10.0.1.13:5432 maxconn 200 check port 8008<\/code><\/pre>\n\n\n\nThe on-marked-down shutdown-sessions<\/code> directive is small but critical. It kills every existing connection to a server the instant that server fails its check, so when the primary goes down, clients are dropped immediately rather than left hanging on a demoted node. Leave it out and an application can keep trying to write to the old primary for the length of its connection timeout. Note also the modern check syntax: HAProxy 2.2 and newer use http-check send meth OPTIONS uri \/primary<\/code> rather than the old one-line option httpchk OPTIONS \/primary<\/code>, and Patroni dropped the legacy \/master<\/code> endpoint in favour of \/primary<\/code>, so older configs you find online will quietly health-check the wrong path.<\/p>\n\n\n\nSELinux on the RHEL family needs one boolean flipped before HAProxy can reach the backends, because the confined HAProxy domain will not open arbitrary outbound ports by default:<\/p>\n\n\n\n
sudo setsebool -P haproxy_connect_any 1<\/code><\/pre>\n\n\n\nOpen the proxy ports and start the service on both nodes:<\/p>\n\n\n\n
sudo firewall-cmd --permanent --add-port={5000,5001,7000}\/tcp && sudo firewall-cmd --reload # RHEL family\nsudo systemctl enable --now haproxy<\/code><\/pre>\n\n\n\nThe stats page at http:\/\/10.0.1.21:7000\/<\/code> shows the routing decision in real time. The leader is green and UP in the postgres_primary<\/code> pool, while the standbys correctly report DOWN there (they answer 503 on \/primary<\/code>) and UP in the postgres_replicas<\/code> pool.<\/p>\n\n\n\n![\"HAProxy](\"https:\/\/computingforgeeks.com\/wp-content\/uploads\/2026\/06\/wm-postgresql-ha-haproxy-stats-page.png\" "\\"\\"")
<\/figure>\n\n\n\nThat red row in the writes pool is not an error. A standby is supposed to fail the \/primary<\/code> check, and HAProxy showing it DOWN there is the routing working as designed. If you set this up with one proxy, you would be done, but the proxy would now be your single point of failure. That is what Keepalived solves.<\/p>\n\n\n\nAdd a floating VIP with Keepalived<\/h2>\n\n\n\n
![\"psql](\"https:\/\/computingforgeeks.com\/wp-content\/uploads\/2026\/06\/wm-postgresql-ha-vip-connection-routing.png\" "\\"\\"")
<\/figure>\n\n\n\n![\"patronictl](\"https:\/\/computingforgeeks.com\/wp-content\/uploads\/2026\/06\/wm-postgresql-ha-automatic-failover.png\" "\\"\\"")
<\/figure>\n\n\n\n