Description

I'm trying to configure an external auth service by following the example from the docs (https://gateway.envoyproxy.io/v1.0.1/tasks/security/ext-auth/). My real setup differs however in some parts:

• my ext auth service is in a namespace different from the namespace of the Gateway resource.
• I need the configuration on the Gateway level and not on the HTTPRoute level. To have a global behavior (each inbound request handled by the particular gateway should be forwarded to the ext auth service regardless of the route)

While the latter can be achieved by the following SecurityPolicy

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: ext-auth-heimdall
  namespace: gw-ns
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
    namespace: gw-ns
  extAuth:
    grpc:
      backendRef:
        name: heimdall
        port: 4456
        namespace: heimdall

it works only if the Gateway resource and the ext auth service are in the same namespace. If they are in different namespaces, as also shown in the snippet above, the status stanza of the security policy tells that the policy is invalid:

Status:
  Ancestors:
    Ancestor Ref:
      Group:      gateway.networking.k8s.io
      Kind:       Gateway
      Name:       eg
      Namespace:  gw-ns
    Conditions:
      Last Transition Time:  2024-05-01T23:36:01Z
      Message:               Backend ref to Service heimdall/heimdall not permitted by any ReferenceGrant
      Observed Generation:   1
      Reason:                Invalid
      Status:                False
      Type:                  Accepted
    Controller Name:         gateway.envoyproxy.io/gatewayclass-controller
Events:                      <none>

Similar reasoning message is also shown in the logs of the gateway controller

2024-05-01T23:23:05.374Z	INFO	provider	kubernetes/controller.go:450	no matching ReferenceGrants found	{"runner": "provider", "from": "HTTPRoute", "from namespace": "gw-ns", "target": "Service", "target namespace": "heimdall"}

Obviously, there is a need for a ReferenceGrant. If I create it (with the contents shown below), the controller logs, that it was able to add the reference grant, but it has no effect. The security policy has still the same status.

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: ReferenceGrant
metadata:
  name: ext-auth-heimdall-rg-hr
  namespace: heimdall
spec:
  from:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      namespace: gw-ns
  to:
    - group: ""
      kind: Service
      name: heimdall

Actually, I would expect, the reference grant is required for the Gateway itself as there is no HTTPRoute resource in place "between" the gateway and the ext auth service. The configuration happens by making use of the extAuth property in the SecurityPolicy. So instead of kind: HTTPRoute , it should be kind: Gateway. Deploying a corresponding reference grant (with kind: Gateway) has however no effect as well.

Do I miss something? Is it maybe even a bug in the controller implementation? Is cross-namespace access for non xRoute ext auth services possible at all? Thank you in advance.

Additional Resources

This issue/challenge/observation has been initially asked in Slack: https://envoyproxy.slack.com/archives/C03E6NHLESV/p1714637933500289

The context of this issue is an integration guide I'm writing for a service I'm maintaining (https://github.com/dadrus/heimdall) and a corresponding example, which you can find under https://github.com/dadrus/heimdall/blob/8927261af52e0e3c2b6b6f35f0d0d8420c17815d/examples/kubernetes/Justfile#L218 (you'll need to build an image for heimdall and load it into the kind cluster if you would like to check it out: just build-image, followed by kind load docker-image heimdall:local -n demo-cluster when the cluster is started). As of now, the gateway and my ext auth server reside in the same namespace to overcome the limitations described above and to have something working. This is however not how this service is typically used in prod.