Skip to content

bpf: migrate branching config macros to runtime configuration #42646

@rgo3

Description

@rgo3

The next step in the clang-freedom (pre-compiled eBPF programs) journey is to migrate compile-time configuration macros that influence control flow to load-time (runtime) configuration. This complements issue #38370 which handles non-branching configuration.

Runtime configuration can be defined with DECLARE_CONFIG and NODE_CONFIG. See Configuring the Datapath in the Cilium Developer docs for guidelines and principles.

This issue is for migrating branching macros (primarily ENABLE_* prefixed) from compile-time to DECLARE_CONFIG/NODE_CONFIG, as well as updating the loader infrastructure to support runtime configuration of these features. Through this issue, we can coordinate work and keep an overview. When migrating variables, do a quick search through Paul's macro documentation for a good docstring to use in DECLARE_CONFIG.

This list is not guaranteed to be comprehensive, may change over time and may grow as we find more work in this category.

Protocol & Address Family Features

Set in WriteNodeConfig() based on option.Config:

  • ENABLE_IPV4 (option.Config.EnableIPv4)
  • ENABLE_IPV6 (option.Config.EnableIPv6)
  • ENABLE_IPV4_FRAGMENTS (option.Config.EnableIPv4FragmentsTracking)
  • ENABLE_IPV6_FRAGMENTS (option.Config.EnableIPv6FragmentsTracking)
  • ENABLE_SCTP (option.Config.EnableSCTP)
  • ENABLE_SRV6 (option.Config.EnableSRv6)
  • ENABLE_SRV6_SRH_ENCAP (option.Config.SRv6EncapMode)

NodePort & Load Balancing

Set in WriteNodeConfig() based on KPR and load balancer configuration:

  • ENABLE_NODEPORT (h.kprCfg.KubeProxyReplacement || option.Config.EnableBPFMasquerade)
  • ENABLE_NODEPORT_ACCELERATION (option.Config.NodePortAcceleration)
  • ENABLE_DSR (cfg.LBConfig.LoadBalancerUsesDSR())
  • ENABLE_DSR_HYBRID (cfg.LBConfig.LBMode or cfg.LBConfig.LBModeAnnotation)
  • ENABLE_DSR_ICMP_ERRORS (option.Config.EnablePMTUDiscovery)
  • ENABLE_DSR_BYUSER (cfg.LBConfig.LBModeAnnotation)
  • DSR_ENCAP_MODE (GENEVE, IPIP - cfg.LBConfig.DSRDispatch)
  • ENABLE_L7_LB (option.Config.EnableEnvoyConfig)
  • ENABLE_TPROXY (option.Config.EnableBPFTProxy)
  • ENABLE_SOCKET_LB_FULL (h.kprCfg.EnableSocketLB && !option.Config.BPFSocketLBHostnsOnly)
  • ENABLE_SOCKET_LB_HOST_ONLY (h.kprCfg.EnableSocketLB && option.Config.BPFSocketLBHostnsOnly)
  • ENABLE_SOCKET_LB_PEER (option.Config.EnableSocketLBPeer)
  • ENABLE_HEALTH_CHECK (option.Config.EnableHealthDatapath)
  • DISABLE_EXTERNAL_IP_MITIGATION (option.Config.DisableExternalIPMitigation)
  • LB_SELECTION_PER_SERVICE (cfg.LBConfig.AlgorithmAnnotation)
  • ENABLE_MKE (option.Config.EnableMKE && h.kprCfg.EnableSocketLB)

NAT & Masquerading

Set in WriteNodeConfig() based on masquerade configuration:

  • ENABLE_MASQUERADE_IPV4 (option.Config.EnableIPv4Masquerade)
  • ENABLE_MASQUERADE_IPV6 (option.Config.EnableIPv6Masquerade)
  • ENABLE_IP_MASQ_AGENT_IPV4 (option.Config.EnableIPMasqAgent)
  • ENABLE_IP_MASQ_AGENT_IPV6 (option.Config.EnableIPMasqAgent)
  • ENABLE_NAT_46X64 (option.Config.NodePortNat46X64)
  • ENABLE_NAT_46X64_GATEWAY (option.Config.EnableNat46X64Gateway)

Network Policy & Security

Set in WriteNodeConfig() or writeTemplateConfig():

  • ENABLE_HOST_FIREWALL (option.Config.EnableHostFirewall)
  • ENABLE_ICMP_RULE (option.Config.EnableICMPRules)
  • ALLOW_ICMP_FRAG_NEEDED (option.Config.AllowICMPFragNeeded)
  • ENABLE_SIP_VERIFICATION (endpoint option)

Encryption

Set via subsystem-specific defines:

Routing & Network Configuration

Set in WriteNodeConfig() or per-endpoint in writeTemplateConfig():

  • ENABLE_ROUTING (per-endpoint: e.RequireRouting())
  • ENABLE_HOST_ROUTING (!option.Config.EnableHostLegacyRouting)
  • ENABLE_ENDPOINT_ROUTES (option.Config.EnableEndpointRoutes)
  • ENABLE_SKIP_FIB (per-endpoint routing logic)
  • TUNNEL_MODE (option.Config.TunnelingEnabled())

Advanced Features

ARP Features

Set per-endpoint in writeTemplateConfig():

  • ENABLE_ARP_RESPONDER (per-endpoint: !e.RequireARPPassthrough())
  • ENABLE_ARP_PASSTHROUGH (per-endpoint: e.RequireARPPassthrough())

Bandwidth Management

Set via pkg/datapath/linux/bandwidth subsystem defines:

  • ENABLE_BANDWIDTH_MANAGER (option.Config.EnableBandwidthManager)

Observability & Debugging

Set per-endpoint via option.IntOptions:

  • DEBUG (per-endpoint mutable option)
  • LB_DEBUG (per-endpoint mutable option)
  • DROP_NOTIFY (per-endpoint mutable option)
  • TRACE_NOTIFY (per-endpoint mutable option)
  • TRACE_SOCK_NOTIFY (option.Config.EnableSocketLBTracing)
  • POLICY_VERDICT_NOTIFY (per-endpoint mutable option)
  • POLICY_AUDIT_MODE (per-endpoint mutable option)

Other Runtime Flags

  • ENABLE_IDENTITY_MARK (option.Config.EnableIdentityMark)
  • ENABLE_JIFFIES (option.Config.ClockSource)
  • CONNTRACK_ACCOUNTING (per-endpoint mutable option)
  • POLICY_ACCOUNTING (per-endpoint mutable option)
  • LOCAL_DELIVERY_METRICS (always enabled for endpoint programs)
  • HOST_ENDPOINT (marks host endpoint programs)
  • USE_BPF_PROG_FOR_INGRESS_POLICY (per-endpoint based on e.RequireEgressProg() or option.Config.EnableEndpointRoutes)
  • SERVICE_NO_BACKEND_RESPONSE (option.Config.ServiceNoBackendResponse)

Notes

  • All macros listed above should actually be set at runtime by the agent based on configuration
  • Macros are often set either in:
    • WriteNodeConfig() in pkg/datapath/linux/config/config.go (node-level)
    • writeTemplateConfig() (per-endpoint)
    • Subsystem-specific NodeExtraDefines (ipsec, wireguard, bandwidth, egress gateway)
    • Per-endpoint IntOptions (debugging and accounting flags)
  • Some macros may be interdependent or mutually exclusive
  • Each migration should include corresponding loader/agent changes
  • Test coverage should ensure both enabled/disabled code paths work correctly, see pkg/datapath/loader/verifier_load_test.go
  • This list was created with help from robots, feel free to call out any config values added here by mistake.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.area/loaderImpacts the loading of BPF programs into the kernel.kind/metaMeta-task for co-ordination.pinnedThese issues are not marked stale by our issue bot.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions