-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Kubernetes network policy for api-server #20550
Copy link
Copy link
Closed
Labels
area/agentCilium agent related.Cilium agent related.area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.Impacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/bugThis is a bug in the Cilium logic.This is a bug in the Cilium logic.sig/policyImpacts whether traffic is allowed or denied based on user-defined policies.Impacts whether traffic is allowed or denied based on user-defined policies.
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
- My service for api server
kubectl describe svc kubernetes -n default
Name: kubernetes
Namespace: default
Labels: component=apiserver
provider=kubernetes
Annotations: <none>
Selector: <none>
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.233.0.1
IPs: 10.233.0.1
Port: https 443/TCP
TargetPort: 6443/TCP
Endpoints: 10.137.16.20:6443,10.137.16.21:6443,10.137.16.22:6443
Session Affinity: None
Events: <none>
- All egress is blocked
Trying to whitelist api-server via Kubernetes Network Policy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-api-server-access-egress
namespace: default
spec:
podSelector: { } # all pods
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 10.137.17.30/32 # IP of K8s Api Server Endpoints
ports:
- protocol: TCP
port: 6443
- to:
- ipBlock:
cidr: 10.137.17.31/32 # IP of K8s Api Server Endpoints
ports:
- protocol: TCP
port: 6443
- to:
- ipBlock:
cidr: 10.137.17.32/32 # IP of K8s Api Server Endpoints
ports:
- protocol: TCP
port: 6443
Trafic is still blocked.
In cilium monitor I see blocked 10.137.17.30:6443 , 10.137.17.31:6443, 10.137.17.32:6443
Policy verdict log: flow 0x17d61b8f local EP ID 2679, remote ID kube-apiserver, proto 6, egress, action deny, match none, 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
xx drop (Policy denied) flow 0x17d61b8f to endpoint 0, , identity 60570->kube-apiserver: 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
Policy verdict log: flow 0x635df349 local EP ID 2679, remote ID kube-apiserver, proto 6, egress, action deny, match none, 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
xx drop (Policy denied) flow 0x635df349 to endpoint 0, , identity 60570->kube-apiserver: 10.0.10.66:46330 -> 10.137.17.31:6443 tcp SYN
Policy verdict log: flow 0xae4297f5 local EP ID 2679, remote ID host, proto 6, ingress, action allow, match L3-Only, 10.0.10.90:53836 -> 10.0.10.66:8080 tcp SYN
When I use Cilium Network Policy
kind: CiliumNetworkPolicy
metadata:
name: cilium-api-server-egress
namespace: default
spec:
endpointSelector:
matchLabels: {} # all pods
egress:
- toEntities:
- kube-apiserver
- toPorts:
- ports:
- port: "6443"
protocol: TCP
Now everything works fine.
My issue is to use only Kubernetes Network Policy for all my clusters (some of them are based on calico from Tigera)
cilium identity list | grep kube-apiserver
reserved:kube-apiserver
cilium identity get 7
ID LABELS
7 reserved:kube-apiserver
reserved:remote-node
Cilium Version
1.11.2
Kernel Version
5.4.182 on Centos 8
Kubernetes Version
1.22.5
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct
- I agree to follow this project's Code of Conduct
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/agentCilium agent related.Cilium agent related.area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.Impacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/bugThis is a bug in the Cilium logic.This is a bug in the Cilium logic.sig/policyImpacts whether traffic is allowed or denied based on user-defined policies.Impacts whether traffic is allowed or denied based on user-defined policies.