Skip to content

Some issues encountered in writing Ipsec E2E tests #1481

@Geaus

Description

@Geaus

I followed the steps below to deploy kmesh with ipsec enabled:
1.kmeshctl secret --key=$(dd if=/dev/urandom count=36 bs=1 2>/dev/null | xxd -p -c 64)
2.kubectl apply -f ./kmesh/deploy/yaml/crd/kmesh.net_kmeshnodeinfos.yaml
3.kubectl apply -f ./kmesh/deploy/yaml/
[ "./start_kmesh.sh --mode=dual-engine --enable-bypass=false --enable-ipsec=true", ]
4.kubectl label namespace default istio.io/dataplane-mode=Kmesh
kubectl apply -f ./kmesh/samples/httpbin/httpbin.yaml with nodeName: kmesh-testing-worker
kubectl apply -f ./kmesh/samples/sleep/sleep.yaml with nodeName: kmesh-testing-control-plane

In the sleep pod,curl ip + port of the httpbin service would fail, while kmesh reported an error:
time="2025-08-12T11:00:52Z" level=error msg="grpc reconnect failed, create workload stream failed, DeltaAggregatedResources failed, rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 10.96.95.184:15012: i/o timeout\"" subsys=controller

curl the ip + port of the httpbin pod would succeed. The ESP package can be seen in the docker container of the kind cluster.

Then I run E2E test script with ipsec enabled. The modifications I made are as follows:
1.add volume and volumount
- mountPath: /kmesh/kmesh-ipsec name: kmesh-ipsec

- name: kmesh-ipsec secret: secretName: kmesh-ipsec optional: true in daemonset.yaml
2.add clusterrole apigroups
- apiGroups: ["kmesh.net"] resources: ["kmeshnodeinfos"] verbs: ["get", "create", "update", "delete", "list", "watch"] in kmesh-rbac.yaml
3.then run
kmeshctl secret --key=$(dd if=/dev/urandom count=36 bs=1 2>/dev/null | xxd -p -c 64)
kubectl apply -f deploy/yaml/crd/kmesh.net_kmeshnodeinfos.yaml
helm install kmesh $ROOT_DIR/deploy/charts/kmesh-helm -n kmesh-system --create-namespace --set deploy.kmesh.image.repository=localhost:5000/kmesh \ --set deploy.kmesh.containers.kmeshDaemonArgs="--mode=dual-engine --enable-bypass=false --monitoring=true --enable-ipsec=true"

Then the waypoint failed to start and kmesh log as:
2025-08-12T10:52:18.598497Z error citadelclient failed to sign CSR: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup istiod.istio-system.svc: i/o timeout"
2025-08-12T10:52:18.598874Z info citadelclient recreated connection
2025-08-12T10:52:18.598907Z error cache resource:default failed to sign: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup istiod.istio-system.svc: i/o timeout"
2025-08-12T10:52:18.598913Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup istiod.istio-system.svc: i/o timeout"

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions