Add config event chan length #8892
Description
What is the problem you're trying to solve
current event chan length is default 1000, there will be channel blocking, can we add a system configuration parameter to customize this value.
Describe the solution you'd like
Add an EventChanlSize parameter to cri.Config.
Additional context
I found a lot of discarding events when looking at this e2e log.
Jul 23 18:56:46 tmp-node-e2e-92a26bb7-cos-97-16919-353-1 containerd[910]: time="2023-07-23T18:56:46.424828104Z" level=debug msg="containerEventsChan is full, discarding event {ContainerId:98bac6c4b73cfd4c647647aa80248208b67de059874d4ddfdd1aa43ea3967e54 ContainerEventType:CONTAINER_STOPPED_EVENT CreatedAt:1690138606424696583 PodSandboxStatus:&PodSandboxStatus{Id:17df7d535a8570b5badc642d8f549e6ba8a404cc2c6898a58873a082313678c0,Metadata:&PodSandboxMetadata{Name:oomkill-target-pod,Uid:17efb6ec-09ab-4eec-8ad6-f953eb1a7466,Namespace:oomkiller-test-8312,Attempt:0,},State:SANDBOX_READY,CreatedAt:1690138604987482380,Network:&PodSandboxNetworkStatus{Ip:10.100.0.147,AdditionalIps:[]*PodIP{},},Linux:&LinuxPodSandboxStatus{Namespaces:&Namespace{Options:&NamespaceOption{Network:POD,Pid:CONTAINER,Ipc:POD,TargetId:,UsernsOptions:nil,},},},Labels:map[string]string{io.kubernetes.pod.name: oomkill-target-pod,io.kubernetes.pod.namespace: oomkiller-test-8312,io.kubernetes.pod.uid: 17efb6ec-09ab-4eec-8ad6-f953eb1a7466,},Annotations:map[string]string{kubernetes.io/config.seen: 2023-07-23T18:56:44.955881675Z,kubernetes.io/config.source: api,},RuntimeHandler:,} ContainersStatuses:[&ContainerStatus{Id:98bac6c4b73cfd4c647647aa80248208b67de059874d4ddfdd1aa43ea3967e54,Metadata:&ContainerMetadata{Name:oomkill-multi-target-container,Attempt:0,},State:CONTAINER_EXITED,CreatedAt:1690138605675227684,StartedAt:1690138605807969879,FinishedAt:1690138606325950236,ExitCode:137,Image:&ImageSpec{Image:registry.k8s.io/e2e-test-images/busybox:1.29-4,Annotations:map[string]string{},},ImageRef:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937,Reason:Error,Message:,Labels:map[string]string{io.kubernetes.container.name: oomkill-multi-target-container,io.kubernetes.pod.name: oomkill-target-pod,io.kubernetes.pod.namespace: oomkiller-test-8312,io.kubernetes.pod.uid: 17efb6ec-09ab-4eec-8ad6-f953eb1a7466,},Annotations:map[string]string{io.kubernetes.container.hash: 9d865647,io.kubernetes.container.restartCount: 0,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},Mounts:[]*Mount{&Mount{ContainerPath:/etc/hosts,HostPath:/var/lib/kubelet/pods/17efb6ec-09ab-4eec-8ad6-f953eb1a7466/etc-hosts,Readonly:false,SelinuxRelabel:false,Propagation:PROPAGATION_PRIVATE,UidMappings:[]*IDMapping{},GidMappings:[]*IDMapping{},},&Mount{ContainerPath:/dev/termination-log,HostPath:/var/lib/kubelet/pods/17efb6ec-09ab-4eec-8ad6-f953eb1a7466/containers/oomkill-multi-target-container/af3cc608,Readonly:false,SelinuxRelabel:false,Propagation:PROPAGATION_PRIVATE,UidMappings:[]*IDMapping{},GidMappings:[]*IDMapping{},},},LogPath:/var/log/pods/oomkiller-test-8312_oomkill-target-pod_17efb6ec-09ab-4eec-8ad6-f953eb1a7466/oomkill-multi-target-container/0.log,Resources:&ContainerResources{Linux:&LinuxContainerResources{CpuPeriod:100000,CpuQuota:0,CpuShares:2,MemoryLimitInBytes:15728640,OomScoreAdj:997,CpusetCpus:,CpusetMems:,HugepageLimits:[]*HugepageLimit{},Unified:map[string]string{memory.oom.group: 1,memory.swap.max: 0,},MemorySwapLimitInBytes:15728640,},Windows:nil,},}] XXX_NoUnkeyedLiteral:{} XXX_sizecache:0}"
- add metrics for dropped events