-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Fail with a clear error message if a Gateway TLS listener has a cross-namespace secret reference #5610
Description
Describe the bug:
When a user specifies a Gateway with a cert-manager gateway shim annotation and a listener block that references a TLS certs secret in a different namespace than the Gateway the certificate provisioning will fail without a clear error message.
Context:
cert-manager can auto-create Certificates for annotated networking.k8s.io Gateway resources, see docs for how this mechanism works.
More recent versions of Gateway support referencing secrets in namespaces different than the Gateway resource itself via Reference Grant mechanism.
cert-manager creates Kubernetes Secrets with the issued TLS certs for a Certificate custom resource in the same namespace as the Certificate. The Certificates for Gateway resources have an owner reference to the Gateway (so that they can be garbage collected when the Gateway gets deleted.) Owner references do not work across namespaces so our existing mechanism does not work with autogenerating Certificates for Gateway listeners that reference secrets in a different namespace.
Currently cert-manager will attempt to create a Certificate in the namespace specified via gateway.spec.listeners.tls.certificateRefs.namespace even if it's different than the Gateway's namespace, but this Certificate will get garbage collected by Kubernetes as it will have a broken cross-namespace owner reference to the Gateway.
Expected behaviour:
cert-manager should not attempt to issue a certificate for a listener block that references a secret in a different namespace.
Instead, it should throw a clear error message.
Alternatives:
We could try to allow issuing cross-namespace certs for Gateways, for example by making the owner references optional and using Gateway API's Reference Grant mechanism to verify that the user who created the Gateway is authorized to create Certificate's in the referenced namespace.
However, it seems like cross-namespace TLS secrets are mostly used in cases where folks have some existing certificates (i.e from a private PKI) that they want to re-use and also want to restrict to just a single namespace, so perhaps this is not important for cert-manager. Would be interesting to hear from users if this is something that would be valuable.
Steps to reproduce the bug:
- From this repo run
make e2e-setup-kindandmake e2e-setupto get cert manager, Contour and Gateway APIs - Apply a
Gatewaywith a cross-namespace secret ref and anIssuer
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: sandbox
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: foo
namespace: sandbox
spec:
selfSigned: {}
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
name: example
namespace: default
annotations:
cert-manager.io/issuer: foo
spec:
gatewayClassName: acmesolver
listeners:
- name: https
hostname: foo.com
port: 443
protocol: HTTPS
allowedRoutes:
namespaces:
from: All
namespace: sandbox # different than the namespace of the Gateway itself
EOF
- Observe that the
Gatewaystatus suggests that aCertificategot created, but there is noCertificateorSecretinsandboxnamespace and a Kubernetes event about a broken owner reference:
$ kubectl get certificate,secret -n sandbox
No resources found in sandbox namespace.
$ kubectl describe gateway example
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CreateCertificate 2m51s cert-manager-gateway-shim Successfully created Certificate "foo"
$ kubectl get events -n sandbox
LAST SEEN TYPE REASON OBJECT MESSAGE
3m53s Warning OwnerRefInvalidNamespace certificate/foo ownerRef [gateway.networking.k8s.io/v1alpha2/Gateway, namespace: sandbox, name: example, uid: 9b212f42-432d-4aa2-9993-48733c6630fb] does not exist in namespace "sandbox"
Environment details::
- Kubernetes version:
- Cloud-provider/provisioner:
- cert-manager version:
- Install method: e.g. helm/static manifests
/kind bug