Skip to content

Kubernetes network policy for api-server #20550

@djkormo

Description

@djkormo

Is there an existing issue for this?

  • I have searched the existing issues

What happened?

  1. My service for api server

kubectl describe svc kubernetes -n default

Name:              kubernetes
Namespace:         default
Labels:            component=apiserver
                   provider=kubernetes
Annotations:       <none>
Selector:          <none>
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.233.0.1
IPs:               10.233.0.1
Port:              https  443/TCP
TargetPort:        6443/TCP
Endpoints:         10.137.16.20:6443,10.137.16.21:6443,10.137.16.22:6443
Session Affinity:  None
Events:            <none>
  1. All egress is blocked

Trying to whitelist api-server via Kubernetes Network Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-api-server-access-egress
  namespace: default
spec:
  podSelector: { } # all pods
  policyTypes:
    - Egress
  egress:
    - to:
        - ipBlock:
            cidr: 10.137.17.30/32 # IP of K8s Api Server Endpoints
      ports:
        - protocol: TCP
          port: 6443
    - to:
        - ipBlock:
            cidr: 10.137.17.31/32 # IP of K8s Api Server Endpoints
      ports:
        - protocol: TCP
          port: 6443
    - to:
        - ipBlock:
            cidr: 10.137.17.32/32 # IP of K8s Api Server Endpoints
      ports:
        - protocol: TCP
          port: 6443  

Trafic is still blocked.
In cilium monitor I see blocked 10.137.17.30:6443 , 10.137.17.31:6443, 10.137.17.32:6443

Policy verdict log: flow 0x17d61b8f local EP ID 2679, remote ID kube-apiserver, proto 6, egress, action deny, match none, 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
xx drop (Policy denied) flow 0x17d61b8f to endpoint 0, , identity 60570->kube-apiserver: 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
Policy verdict log: flow 0x635df349 local EP ID 2679, remote ID kube-apiserver, proto 6, egress, action deny, match none, 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
xx drop (Policy denied) flow 0x635df349 to endpoint 0, , identity 60570->kube-apiserver: 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
Policy verdict log: flow 0xae4297f5 local EP ID 2679, remote ID host, proto 6, ingress, action allow, match L3-Only, 10.0.10.90:53836 -> 10.0.10.66:8080 tcp SYN

When I use Cilium Network Policy

kind: CiliumNetworkPolicy
metadata:
  name: cilium-api-server-egress
  namespace: default
spec:
  endpointSelector:
    matchLabels: {} # all pods
  egress:
  - toEntities:
    - kube-apiserver
  - toPorts:
    - ports:
      - port: "6443"
        protocol: TCP

Now everything works fine.

My issue is to use only Kubernetes Network Policy for all my clusters (some of them are based on calico from Tigera)

cilium identity list | grep kube-apiserver

reserved:kube-apiserver

cilium identity get 7

ID   LABELS
7    reserved:kube-apiserver
     reserved:remote-node

Cilium Version

1.11.2

Kernel Version

5.4.182 on Centos 8

Kubernetes Version

1.22.5

Sysdump

No response

Relevant log output

No response

Anything else?

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

Labels

area/agentCilium agent related.area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/bugThis is a bug in the Cilium logic.sig/policyImpacts whether traffic is allowed or denied based on user-defined policies.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions