Followup of YunoHost/yunohost#1432
The "real" solution (besides "manually checking every app" ...) seems to require /headers-more-nginx-module version 0.35~0.36 which will only be available in debian trixie .. or maybe bookworm backports, specifically the -a option for more_set_headers which means "not overriding what the proxy would set but only append an additional header with the same name ...
openresty/headers-more-nginx-module@db45361
https://packages.debian.org/bookworm/libnginx-mod-http-headers-more-filter