Proposal / RFE

The proposal is to add a new CRDs that defines CiliumClusterwideCIDRSet and CiliumCIDRSet objects. CiliumCIDRSet or CiliumClusterwideCIDRSet would be referenced from Cilium(Clusterwide)NetworkPolicy objects in egress and/or ingress.

Use Cases

Background:

We run on-prem Kubernetes clusters. We run BGP-peered without masquerading, so pods/services are directly accessible. Our default clusterwide policy is deny all both ingress and egress with few clusterwide exceptions (ICMP, DNS, K8s API access)

Use Case 1:

We keep address books of CIDR ranges today within our legacy firewalls for different classes of networks, such as "corporate desktops and SSL VPNs", "non-production US networks", "production UK networks", etc. We have been moving services to Kubernetes where hosts in these networks will need access to Kubernetes services/pods.

It is awkward to ask each team within Kubernetes to independently maintain duplicates of this list in each network policy when they desire to make their pod available to any of these network groups. The list of networks can get quite large, and the subnet list changes as part of our normal network operations.

By having a centralized CIDRSet object, we can put ownership on networking to maintain this object, and various app teams can be assured that by referencing this object, their pods/services will be accessible by the complete set of desired networks.

Expanding upon the L3 example for CIDRSets in the Cilium documentation, an example of a clusterwide CIDRSet may look something like this:

apiVersion: "cilium.io/v2"
kind: CiliumClusterwideCIDRSet
metadata:
  name: "corporate-networks"
spec:
  CIDRSet:
  - cidr: 10.20.0.0/16
  - cidr: 10.30.0.0/16
    except:
    - 10.30.20.0/24
  - cidr: 192.168.0.0/24
---
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "cidr-rule"
spec:
  endpointSelector:
    matchLabels:
      app: myService
  egress:
  - toCIDR:
    - 20.1.1.1/32
  - toCIDRSet:
    - clusterwideRef: "corporate-networks"
    - cidr: 10.0.0.0/8
      except:
      - 10.96.0.0/12

Use Case 2:

Many of our applications are partially dockerized-hybrid, by which I mean that some parts of the application are running pods, and others parts are in legacy VMs.

In some cases, multiple pods need to reach a central service, such as IBM MQ.

It would simplify maintenance of network policies to create namespaced CiliumCIDRSet objects and reference those across all pod network policies:

apiVersion: "cilium.io/v2"
kind: CiliumCIDRSet
metadata:
  name: "mq-server"
spec:
  CIDRSet:
  - cidr: 10.243.111.101/32
---
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "app-pod-type1"
spec:
  endpointSelector:
    matchLabels:
      app: pod-type1
  egress:
  - toCIDRSet:
    - ref: "mq-server"
---
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "app-pod-type2"
spec:
  endpointSelector:
    matchLabels:
      app: pod-type2
  egress:
  - toCIDRSet:
    - ref: "mq-server"