Feat: support no traffic disruption during kmesh upgrade by 072020127 · Pull Request #1503 · kmesh-net/kmesh

072020127 · 2025-09-11T09:14:55Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Implemented logic for storing and reading persistent MapSpecs
Added retrieval of the current MapSpec
Implemented comparison logic for Map structure
Added unit tests and e2e test programs

Which issue(s) this PR fixes:

Fixes #1409

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

gemini-code-assist

Summary of Changes

Hello @072020127, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant feature enabling seamless, traffic-undisrupted upgrades for the kmesh-daemon. It achieves this by persisting BPF map specifications, comparing them during daemon restarts, and intelligently migrating map data to new versions. This ensures that ongoing network traffic is not interrupted when the daemon is updated.

Highlights

Persistent BPF MapSpecs: Implemented logic for storing and reading persistent BPF MapSpecs, which are crucial for maintaining state across kmesh-daemon upgrades.
MapSpec Comparison Logic: Added retrieval of the current MapSpec and implemented robust comparison logic for BPF Map structures to detect changes between versions.
Seamless Daemon Upgrades: Introduced an 'Update' mode for kmesh-daemon, allowing BPF programs and maps to be updated without traffic disruption by migrating existing map data.
Comprehensive Testing: Included new unit tests for MapSpec loading and comparison, as well as an end-to-end test to validate the traffic-undisrupted upgrade process.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

BPF maps persist, Upgrades flow, no traffic lost, Code ensures smooth ride.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a significant feature for non-disruptive upgrades of the kmesh-daemon. The core logic involves persisting BPF map specifications (MapSpecs), comparing them across versions, and migrating maps when their structure changes. The implementation adds new files for handling BPF updates and loading MapSpecs, along with extensive unit and e2e tests.

My review focuses on the new update logic. I've found a few critical issues related to error handling that could lead to panics, and a bug where changes in struct bitfield sizes are not detected, potentially causing data corruption. I've also identified areas with unhandled errors and code duplication that should be addressed to improve robustness and maintainability. The suggestions provided aim to fix these issues and refactor the code for better clarity.

gemini-code-assist · 2025-09-11T09:17:51Z

pkg/bpf/restart/bpf_update.go

+		if map_old.Offset != uint32(map_new.Offset) {
+            log.Info(map_old.BitfieldSize,  uint32(map_new.BitfieldSize))
+			diff.OffsetChanged = true
+			break
+		}


The BitfieldSize of a struct member is not being compared. This is a critical bug, as a change in bitfield size alters the struct layout and should trigger a map migration to avoid data corruption. The existing log.Info call should be replaced with a proper comparison.

if map_old.Offset != uint32(map_new.Offset) || map_old.BitfieldSize != uint32(map_new.BitfieldSize) { diff.OffsetChanged = true break }

gemini-code-assist · 2025-09-11T09:17:51Z

pkg/bpf/bpf.go

+	mapspec, _ := restart.LoadCompileTimeSpecs(config)
+	restart.SnapshotSpecsByPkg(mapspec)


The error returned by restart.LoadCompileTimeSpecs is ignored. This could lead to restart.SnapshotSpecsByPkg being called with a nil mapspec, which might cause a panic. The error should be checked and handled to prevent this.

Suggested change

mapspec, _ := restart.LoadCompileTimeSpecs(config)

restart.SnapshotSpecsByPkg(mapspec)

mapspec, err := restart.LoadCompileTimeSpecs(config)

if err != nil {

log.Errorf("failed to load compile time specs: %v", err)

return nil

}

restart.SnapshotSpecsByPkg(mapspec)

gemini-code-assist · 2025-09-11T09:17:51Z

pkg/bpf/restart/bpf_update.go

+	specsByPkg, err := LoadCompileTimeSpecs(config)
+	if specsByPkg == nil {
+		log.Errorf("load oldSpecsByPkg failed")
+        return nil
+	}


The error returned by LoadCompileTimeSpecs is not handled. If an error occurs, specsByPkg could be nil and the function might proceed, potentially causing a panic later. It's crucial to check and handle this error. Also, the log message on line 68 refers to oldSpecsByPkg which seems to be a typo.

Suggested change

specsByPkg, err := LoadCompileTimeSpecs(config)

if specsByPkg == nil {

log.Errorf("load oldSpecsByPkg failed")

return nil

}

specsByPkg, err := LoadCompileTimeSpecs(config)

if err != nil {

log.Errorf("load compile-time map specs failed: %v", err)

return nil

}

gemini-code-assist · 2025-09-11T09:17:51Z

pkg/bpf/restart/bpf_update.go

+func migrateMap(
+    oldMapSpec *PersistedMapSpec,
+    newMapSpec *ebpf.MapSpec,
+    pkgName, mapName, pinPath string,
+) (*ebpf.Map, error)  {
+	if oldMapSpec == nil {
+		return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+	}
+    if oldMapSpec.Type != newMapSpec.Type.String() ||
+       oldMapSpec.KeySize != newMapSpec.KeySize ||
+       oldMapSpec.ValueSize != newMapSpec.ValueSize || 
+       oldMapSpec.MaxEntries != newMapSpec.MaxEntries {
+       oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+        if err == nil {
+            return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+        } else {
+            return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+        }
+	}
+
+	oldKeyStruct := oldMapSpec.KeyInfo
+    newKeyStruct, okNewKey := newMapSpec.Key.(*btf.Struct)
+	if okNewKey {
+        diff := diffStructInfoAgainstBTF(oldKeyStruct, newKeyStruct, make(map[string]bool))
+        if diff.Added || diff.Removed || diff.TypeChanged ||
+            diff.OffsetChanged || diff.NestedChanged {
+			oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+            if err == nil {
+                return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+            } else {
+                return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+            }
+		}
+    } else { 
+        if newMapSpec.Key == nil {
+            if len(oldKeyStruct.Members) != 0 || oldKeyStruct.Name != "" {
+                oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+                if err == nil {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+                } else {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+                }
+            }
+        } else {
+            newKeyTypeName := newMapSpec.Key.TypeName()
+            if len(oldKeyStruct.Members) != 0 || newKeyTypeName != oldKeyStruct.Name  {
+                oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+                if err == nil {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+                } else {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+                }
+            }
+        }
+    }
+
+    oldValueStruct := oldMapSpec.ValueInfo
+    newValueStruct, okNewValue := newMapSpec.Value.(*btf.Struct)
+    if okNewValue {
+        diff := diffStructInfoAgainstBTF(oldValueStruct, newValueStruct, make(map[string]bool))
+        if diff.Added || diff.Removed || diff.TypeChanged ||
+            diff.OffsetChanged || diff.NestedChanged {
+            oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+            if err == nil {
+                return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+            } else {
+                log.Info(diff.Added ,diff.Removed,diff.TypeChanged ,
+            diff.OffsetChanged,diff.NestedChanged)
+                return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+            }
+        }
+    } else {
+        if newMapSpec.Value == nil {
+            if len(oldValueStruct.Members) != 0 || oldValueStruct.Name != "" {
+                oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+                if err == nil {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+                } else {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+                }
+            }
+        } else {
+            newValTypeName := newMapSpec.Value.TypeName()
+            if len(oldValueStruct.Members) != 0 || newValTypeName != oldValueStruct.Name  {
+                oldMap, err := ebpf.LoadPinnedMap(pinPath, &ebpf.LoadPinOptions{})
+                if err == nil {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, oldMap)
+                } else {
+                    return createEmptyMap(newMapSpec, pinPath, mapName, nil)
+                }
+            }
+        }
+    }
+    log.Info("pass all check")
+	return nil, nil
+}


The migrateMap function has high complexity and significant code duplication. The logic to decide whether to recreate a map is repeated in multiple branches. This makes the code hard to read and maintain. Consider refactoring this function to extract the map recreation logic into a helper function and simplify the conditional checks.

codecov · 2025-09-16T03:25:36Z

Codecov Report

❌ Patch coverage is 0.53619% with 371 lines in your changes missing coverage. Please review.
✅ Project coverage is 37.81%. Comparing base (716c0d3) to head (3c861d1).
⚠️ Report is 208 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/bpf/restart/bpf_update.go	0.00%	263 Missing ⚠️
pkg/bpf/restart/new_version_mapspec_loader.go	0.00%	95 Missing ⚠️
pkg/bpf/bpf.go	20.00%	6 Missing and 2 partials ⚠️
pkg/bpf/ads/sock_connection.go	0.00%	1 Missing ⚠️
pkg/bpf/ads/sock_ops.go	0.00%	1 Missing ⚠️
pkg/bpf/workload/skb.go	0.00%	1 Missing ⚠️
pkg/bpf/workload/sock_connection.go	0.00%	1 Missing ⚠️
pkg/bpf/workload/sock_ops.go	0.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
pkg/bpf/ads/sock_connection.go	`0.00% <0.00%> (ø)`
pkg/bpf/ads/sock_ops.go	`0.00% <0.00%> (ø)`
pkg/bpf/workload/skb.go	`0.00% <0.00%> (ø)`
pkg/bpf/workload/sock_connection.go	`0.00% <0.00%> (ø)`
pkg/bpf/workload/sock_ops.go	`0.00% <0.00%> (ø)`
pkg/bpf/bpf.go	`45.48% <20.00%> (+5.05%)`	⬆️
pkg/bpf/restart/new_version_mapspec_loader.go	`0.00% <0.00%> (ø)`
pkg/bpf/restart/bpf_update.go	`0.00% <0.00%> (ø)`

... and 93 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7da837...3c861d1. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

hzxuzhonghu

I am concerned this is a very big change, could introduce regression. Can you please add a feature flag, we can turn on until we have enough tests and documents

hzxuzhonghu · 2025-09-16T08:11:53Z

pkg/bpf/restart/new_version_mapspec_loader.go

+
+// LoadCompileTimeSpecs loads all compile-time MapSpecs from bpf2go-generated packages.
+// Returns a nested map: packageName to mapName to *ebpf.MapSpec
+func LoadCompileTimeSpecs(config *options.BpfConfig) (map[string]map[string]*ebpf.MapSpec, error) {


what do you mean by CompileTime

My intention is to describe that these specs are fixed when the binary is compiled (via bpf2go), as opposed to the specs loaded from the persisted on-disk snapshot. Maybe LoadEmbeddedSpecs will be clearer to reflect its role?

hzxuzhonghu · 2025-09-16T08:14:43Z

pkg/bpf/restart/persisted_mapspec_loader.go

+		} else {
+			specs["KmeshCgroupSockWorkload"] = coll.Maps
+		}
+		if coll, err := dualengine.LoadKmeshCgroupSockWorkloadCompat(); err != nil {


we should only call either LoadKmeshCgroupSockWorkload or LoadKmeshCgroupSockWorkloadCompat

please check

if helper.KernelVersionLowerThan5_13() { spec, err = bpf2go.LoadKmeshSockopsCompat() } else { spec, err = bpf2go.LoadKmeshSockops() }

hzxuzhonghu · 2025-09-16T08:17:51Z

pkg/bpf/restart/new_version_mapspec_loader.go

+
+// LoadCompileTimeSpecs loads all compile-time MapSpecs from bpf2go-generated packages.
+// Returns a nested map: packageName to mapName to *ebpf.MapSpec
+func LoadCompileTimeSpecs(config *options.BpfConfig) (map[string]map[string]*ebpf.MapSpec, error) {


I am wondering how we can not miss one map, even after we add a new map later.

I wrote a go file for gen-bpf_stpecs.go in the/back directory, which can automatically generate LoadCompileTimeSpecs by analyzing the bpf2Go.go file. If there are new maps or progs in the future, it can be easily kept synchronized.

hzxuzhonghu · 2025-09-16T08:19:22Z

pkg/bpf/restart/new_version_mapspec_loader_enhanced.go

+func LoadCompileTimeSpecs(config *options.BpfConfig) (map[string]map[string]*ebpf.MapSpec, error) {
+	specs := make(map[string]map[string]*ebpf.MapSpec)
+
+	if config.KernelNativeEnabled() {


With the tag //go:build enhanced, it is for kernel native mode, so we donot need the below else branch

hzxuzhonghu · 2025-09-16T08:19:58Z

pkg/bpf/bpf.go

 		return nil
 	}

+	mapspec, err := restart.LoadCompileTimeSpecs(config)


Suggested change

mapspec, err := restart.LoadCompileTimeSpecs(config)

mapSpecs, err := restart.LoadCompileTimeSpecs(config)

hzxuzhonghu · 2025-09-16T08:39:58Z

pkg/bpf/restart/bpf_update.go

+// It will migrate any BPF maps whose on‑disk pin already exists but whose
+// compiled MapSpec has changed
+func UpdateMapHandler(versionMap *ebpf.Map, kmBpfPath string, config *options.BpfConfig) *ebpf.Map {
+	persistedSpecs, err := LoadPersistedSnapshot()


persistedSpecs can be nil, which cause panic later, please check all

hzxuzhonghu · 2025-09-16T08:43:51Z

pkg/bpf/restart/bpf_update.go

+					log.Warnf("failed to remove old map pinpath: %v (continuing)", err)
+				}
+			case hasNew && !hasOld:
+				if _, err := migrateMap(nil, newSpec, pkgName, mapName, pinPath); err != nil {


Would prefer directly use call the map create function

hzxuzhonghu · 2025-09-16T08:45:34Z

pkg/bpf/restart/bpf_update.go

+	return versionMap
+}
+
+func updateVersionInfo(versionMap *ebpf.Map) {


we can share with that one storeVersionInfo in pkg/bpf/bpf.go

I need to import package restart in pkg/bpf/bpf.go. If i call storeVersionInfo in pkg/bpf/bpf.go, this will result in recursive import.

hzxuzhonghu · 2025-09-16T08:51:18Z

test/e2e/upgarde_test.go

+}
+
+// patchKmesh applies a strategic merge patch to the Kmesh DaemonSet and waits for rollout completion.
+func patchKmesh_upgarde(t framework.TestContext, patchData string) {


typo upgrade

hzxuzhonghu · 2025-09-16T08:52:30Z

test/e2e/upgarde_test.go

+func upgradeKmesh(t framework.TestContext) {
+	newImage := os.Getenv("KMESH_UPGRADE_IMAGE")
+	if newImage == "" {
+		newImage = "localhost:5000/kmesh"


isnot this same image with old daemonset?

I attached a new image which is build in run_test.sh set_daemonupgarde_testcase_image() to KMESH_UPGRADE_IMAGE. But E2e test istio 1.23~1.25 report no space left in device. I think the reason is there are two images existing ((localhost:5000/kmesh) and localhost:5000/kmesh/test-upgrade-map-change)

LiZhenCheng9527 · 2025-10-15T02:17:51Z

We update configuration of e2e to fix this errors. Please rebase and then resubmit the code.

Signed-off-by: 072020127 <mhy200253@gmail.com>

hzxuzhonghu · 2025-10-29T02:03:03Z

lgtm

please fix the ci

Signed-off-by: 072020127 <mhy200253@gmail.com>

072020127 · 2025-10-29T03:35:03Z

The CI is frequently getting stuck on the eBPF unit test section. However, the code in this PR does not modify any eBPF-related modules, and the eBPF unit tests are able to pass locally. I have re-tested 4 times. In those 4 tests, the eBPF unit tests ran for 1 hour twice, and reported a "no space left" error the other two times. Therefore, I think it may be a problem with the CI environment

LiZhenCheng9527 · 2025-10-30T06:24:45Z

This issue was caused by using the Ubuntu 22.04 image version in CI. However, it has been resolved by upgrading the Ubuntu image version.

#1513

So maybe you update the Ubuntu image in CI can solve this issue

Signed-off-by: 072020127 <mhy200253@gmail.com>

LiZhenCheng9527 · 2025-10-30T09:01:12Z

/lgtm
/approve

kmesh-bot · 2025-10-30T09:01:20Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LiZhenCheng9527

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [LiZhenCheng9527]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kmesh-bot added the kind/feature label Sep 11, 2025

kmesh-bot requested review from Okabe-Rintarou-0 and YaoZengzeng September 11, 2025 09:14

kmesh-bot added the size/XXL label Sep 11, 2025

gemini-code-assist bot reviewed Sep 11, 2025

View reviewed changes

072020127 force-pushed the main branch from 9d71daf to 7ea199f Compare September 15, 2025 03:42

kmesh-bot added the do-not-merge/contains-merge-commits label Sep 15, 2025

072020127 force-pushed the main branch from 3e5f5f3 to 59ecb57 Compare September 15, 2025 07:03

kmesh-bot added do-not-merge/contains-merge-commits and removed do-not-merge/contains-merge-commits labels Sep 15, 2025

072020127 force-pushed the main branch from df31519 to 843f6bb Compare September 16, 2025 07:20

kmesh-bot removed the do-not-merge/contains-merge-commits label Sep 16, 2025

hzxuzhonghu reviewed Sep 16, 2025

View reviewed changes

072020127 force-pushed the main branch from 843f6bb to d39b94d Compare September 21, 2025 09:51

kmesh-bot added the do-not-merge/contains-merge-commits label Sep 21, 2025

072020127 force-pushed the main branch 4 times, most recently from a21a29c to aee4427 Compare September 21, 2025 11:08

kmesh-bot added do-not-merge/contains-merge-commits and removed do-not-merge/contains-merge-commits labels Sep 21, 2025

072020127 force-pushed the main branch from 8e70a2d to f222714 Compare September 26, 2025 02:24

kmesh-bot removed the do-not-merge/contains-merge-commits label Sep 26, 2025

072020127 force-pushed the main branch 3 times, most recently from ba67a19 to 5ffbc87 Compare September 29, 2025 02:25

kmesh-bot added the do-not-merge/contains-merge-commits label Oct 15, 2025

072020127 added 15 commits October 28, 2025 15:57

golangci-lint

d640566

Signed-off-by: 072020127 <mhy200253@gmail.com>

modify bpf_test.go

6cebd5b

Signed-off-by: 072020127 <mhy200253@gmail.com>

deal with ut

541fad7

Signed-off-by: 072020127 <mhy200253@gmail.com>

pkg/bpf/bpf_test.go

2442122

Signed-off-by: 072020127 <mhy200253@gmail.com>

add auto generate map spec loader and fix bug

f5ecde7

Signed-off-by: 072020127 <mhy200253@gmail.com>

add auto generate map spec loader and fix bug

79a075a

Signed-off-by: 072020127 <mhy200253@gmail.com>

add copy right

82e2f35

Signed-off-by: 072020127 <mhy200253@gmail.com>

fix Makefile

b455214

Signed-off-by: 072020127 <mhy200253@gmail.com>

fix os import

48d2c32

Signed-off-by: 072020127 <mhy200253@gmail.com>

fix upgrade test

7c388f0

Signed-off-by: 072020127 <mhy200253@gmail.com>

e2e test add new map

5ba67dd

Signed-off-by: 072020127 <mhy200253@gmail.com>

add new test case

d2734f6

Signed-off-by: 072020127 <mhy200253@gmail.com>

fix run test

2d952f4

Signed-off-by: 072020127 <mhy200253@gmail.com>

rename mapspce loader files

694b7d0

Signed-off-by: 072020127 <mhy200253@gmail.com>

add flag skip build update image

e374d1c

Signed-off-by: 072020127 <mhy200253@gmail.com>

072020127 force-pushed the main branch from 9a0e318 to e374d1c Compare October 28, 2025 07:57

072020127 added 2 commits October 29, 2025 10:20

Trigger CI: Rerunning tests

37a53af

Signed-off-by: 072020127 <mhy200253@gmail.com>

Trigger CI: Rerunning tests

4d8e46c

Signed-off-by: 072020127 <mhy200253@gmail.com>

072020127 added 2 commits October 30, 2025 15:15

upgdate the Ubuntu image from 22.04 to 24.04

871e4cf

Signed-off-by: 072020127 <mhy200253@gmail.com>

fix gen check

3c861d1

Signed-off-by: 072020127 <mhy200253@gmail.com>

072020127 force-pushed the main branch from e4d3ff5 to 3c861d1 Compare October 30, 2025 07:50

kmesh-bot assigned LiZhenCheng9527 Oct 30, 2025

kmesh-bot added the lgtm label Oct 30, 2025

kmesh-bot added the approved label Oct 30, 2025

kmesh-bot merged commit f4d1eea into kmesh-net:main Oct 30, 2025
14 of 15 checks passed

		mapspec, _ := restart.LoadCompileTimeSpecs(config)
		restart.SnapshotSpecsByPkg(mapspec)

	mapspec, err := restart.LoadCompileTimeSpecs(config)
	mapSpecs, err := restart.LoadCompileTimeSpecs(config)

Conversation

072020127 commented Sep 11, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

hzxuzhonghu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiZhenCheng9527 commented Oct 15, 2025

Uh oh!

hzxuzhonghu commented Oct 29, 2025

Uh oh!

072020127 commented Oct 29, 2025

Uh oh!

LiZhenCheng9527 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LiZhenCheng9527 commented Oct 30, 2025

Uh oh!

kmesh-bot commented Oct 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

codecov bot commented Sep 16, 2025 •

edited

Loading

LiZhenCheng9527 commented Oct 30, 2025 •

edited

Loading