[Feature]: support changing image and PostgreSQL settings simultaneously #2530
Description
Updating the shared preload libraries and the imageName at the same time, on clusters with one or more replicas and with primaryUpdateMethod: switchover can take the cluster in a broken state.
This happens when the new shared preload libraries adds values referring to pg extensions that are available on the new image, but they are not in the old one.
Looks like that the operator starts updating the replicas first, which goes fine. When it comes to update the primary, the switchover happens and the old-primary pod starts getting updated. This last pod update is done using the old image along with the new configuration, so if a shared preload libraries's item in the new config is not installed in the old image, the pod start crashing saying it can't find the extension file.
Example:
NAME AGE INSTANCES READY STATUS PRIMARY
cluster-example 4h17m 3 2 Upgrading cluster cluster-example-1
NAME READY STATUS RESTARTS AGE
cluster-example-1 1/1 Running 0 33m
cluster-example-2 0/1 CrashLoopBackOff 11 (44s ago) 39m
cluster-example-3 1/1 Running
k get pod cluster-example-1 -o yaml | yq .spec.containers[0].image
***/postgres:15.3-9-debian --> new image
k get pod cluster-example-2 -o yaml | yq .spec.containers[0].image
***/postgres:15.3-7-debian --> old image
k logs cluster-example-2
...
{"level":"info","ts":"2023-08-02T14:46:44Z","logger":"postgres","msg":"2023-08-02 14:46:44.655 UTC [28] FATAL: could not access file \"autocluster\": No such file or directory","pipe":"stderr","logging_pod":"cluster-example-2"}
...