In the event of an etcd connectivity loss spanning into the renewal period of a session, Cilium can get into a situation where the session is constantly being renewed:
Jul 22 11:32:13 k8s1 cilium-agent[21715]: level=info msg="Got new lock lease ID 29c67371e2998a5e" config=/var/lib/cilium/etcd-config.yml endpoints="[https://192.168.33.11:2379]" subsys=kvstore
Jul 22 11:32:13 k8s1 cilium-agent[21715]: level=info msg="Renewing etcd lock session" config=/var/lib/cilium/etcd-config.yml endpoints="[https://192.168.33.11:2379]" session="&{0xc000af61a0 0xc00012d520 3010220333913967198 0x497220 0xc000b20840}" subsys=kvstore
Jul 22 11:32:13 k8s1 cilium-agent[21715]: level=info msg="Got new lock lease ID 29c67371e2998a60" config=/var/lib/cilium/etcd-config.yml endpoints="[https://192.168.33.11:2379]" subsys=kvstore
Jul 22 11:32:13 k8s1 cilium-agent[21715]: level=info msg="Renewing etcd lock session" config=/var/lib/cilium/etcd-config.yml endpoints="[https://192.168.33.11:2379]" session="&{0xc000af61a0 0xc0002cb8e0 3010220333913967200 0x497220 0xc000b212c0}" subsys=kvstore
This is happening because the context passed into NewSesssion attempting to enforce a timeout is cancelled after the successful creation of the session. Unfortunately, etcd not only binds blocking operations performed by NewSession to the context but also the lifetime of the resulting lease to it.
In the event of an etcd connectivity loss spanning into the renewal period of a session, Cilium can get into a situation where the session is constantly being renewed:
This is happening because the context passed into
NewSesssionattempting to enforce a timeout is cancelled after the successful creation of the session. Unfortunately, etcd not only binds blocking operations performed by NewSession to the context but also the lifetime of the resulting lease to it.