Conversation
There was a problem hiding this comment.
Summary of Changes
Hello @wxnzb, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request focuses on enhancing the test coverage for BPF programs by introducing a dedicated unit test for the sendmsg functionality. The new test validates the BPF program's ability to intercept and modify network messages, ensuring that it correctly embeds predefined metadata into the message payload. This improves the robustness and reliability of the BPF codebase.
Highlights
- New BPF Map for Socket Management: I've introduced a new
BPF_MAP_TYPE_SOCKHASHmap namedmap_of_kmesh_socketinbpf/kmesh/workload/sendmsg.c. This map is designed to store socket-related information, keyed bystruct bpf_sock_tuple, and will be crucial for managing socket states within BPF programs. - Introduction of
sendmsgBPF Unit Test: I've added a new unit test specifically for thesendmsgBPF program. This involves creatingtest/bpf_ut/workload_sendmsg_test.cwhich includes the originalsendmsg.cwith a mockedbpf_sk_storage_gethelper, allowing for isolated testing of thesendmsglogic. - Comprehensive Go Test Integration: The new
testSendMsgfunction has been integrated intotest/bpf_ut/bpftest/workload_test.go. This Go test sets up a simulated network environment, attaches thesendmsg_progto asockhashmap, and verifies that the BPF program correctly modifies outgoing messages by encoding specific metadata (like IP and port) into the payload using a TLV (Type-Length-Value) format.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request introduces a unit test for the sendmsg BPF program. The changes include adding the BPF test file, Go test logic, and Makefile rules. The review identifies a critical issue with a BPF map name mismatch that would cause the test to fail, a high-severity issue with a potential race condition in the test, and several medium-severity issues related to maintainability and code clarity, such as duplicated flags in the Makefile and use of magic numbers.
bpf/kmesh/workload/sendmsg.c
Outdated
| struct { | ||
| __uint(type, BPF_MAP_TYPE_SOCKHASH); | ||
| __type(key, struct bpf_sock_tuple); | ||
| __type(value, __u32); | ||
| __uint(max_entries, MAP_SIZE_OF_MANAGER); | ||
| __uint(map_flags, 0); | ||
| } map_of_kmesh_socket SEC(".maps"); |
There was a problem hiding this comment.
The BPF map is named map_of_kmesh_socket, but the Go test code in test/bpf_ut/bpftest/workload_test.go:806 attempts to access it using the name km_socket. This mismatch will cause the test to fail with a nil map object when trying to get its file descriptor.
To ensure consistency with other BPF maps accessed from Go tests in this file (e.g., km_manage) and to fix the test, it's recommended to rename this map to km_socket.
struct {
__uint(type, BPF_MAP_TYPE_SOCKHASH);
__type(key, struct bpf_sock_tuple);
__type(value, __u32);
__uint(max_entries, MAP_SIZE_OF_MANAGER);
__uint(map_flags, 0);
} km_socket SEC(".maps");There was a problem hiding this comment.
Yes, why define a map here, we already have one in bpf/kmesh/workload/sockops.c
There was a problem hiding this comment.
We need to redefine the map in sendmsg.c because each eBPF object file (.o) only contains its own maps. The Go test loads workload_sendmsg_test.o, so it can only access maps defined in that file. Even though the same map exists in sockops.c, it’s in a different ELF file, so the test can’t see it.
By redefining it as km_socket in sendmsg.c, the map exists in the test’s ELF file and the Go test can access it.
I think this solution works for the unit test. Do you think it’s reasonable, or do you have a better approach?
| t.Fatalf("Failed to start TCP server: %v", err) | ||
| } | ||
| defer listener.Close() | ||
| go func() { |
There was a problem hiding this comment.
The test starts a server in a goroutine but doesn't wait for it to finish. This can lead to flaky tests where the main test function exits before the goroutine has completed its checks, potentially missing errors reported by t.Error or t.Errorf.
To fix this, you should use a sync.WaitGroup:
- Add
var wg sync.WaitGroupandwg.Add(1)before starting the goroutine. - Add
defer wg.Done()at the beginning of the goroutine. - Add
wg.Wait()at the end of theworkFunc, after the client has sent data, to wait for the server goroutine to finish.
test/bpf_ut/Makefile
Outdated
| WORKLOAD_SENDMSG_FLAGS = -I$(ROOT_DIR)/bpf/kmesh/ -I$(ROOT_DIR)/bpf/kmesh/workload/include -I$(ROOT_DIR)/api/v2-c | ||
| workload_sendmsg_test.o: workload_sendmsg_test.c | ||
| $(QUIET) $(CLANG) $(CLANG_FLAGS) $(WORKLOAD_SENDMSG_FLAGS) -c $< -o $@ |
There was a problem hiding this comment.
| clientPort := 12345 | ||
| serverPort := 54321 | ||
| serverSocket := localIP + ":" + strconv.Itoa(serverPort) |
test/bpf_ut/bpftest/workload_test.go
Outdated
| if n < 16 { | ||
| t.Errorf("Received data too short for TLV format") | ||
| return | ||
| } | ||
| if buf[0] != 0x01 { | ||
| t.Errorf("Unexpected TLV Type: got %x, want 0x01", buf[0]) | ||
| } | ||
| length := binary.BigEndian.Uint32(buf[1:5]) | ||
| if length != 6 { | ||
| t.Errorf("Unexpected TLV Length: got %d, want 6", length) | ||
| } | ||
|
|
||
| ip := net.IPv4(buf[5], buf[6], buf[7], buf[8]) | ||
| expectedIP := net.IPv4(8, 8, 8, 8) | ||
| if !ip.Equal(expectedIP) { | ||
| t.Errorf("Unexpected TLV IP: got %v, want %v", ip, expectedIP) | ||
| } | ||
|
|
||
| port := binary.BigEndian.Uint16(buf[9:11]) | ||
| if port != 53 { | ||
| t.Errorf("Unexpected TLV Port: got %d, want 53", port) | ||
| } | ||
|
|
||
| if buf[11] != 0xFE { | ||
| t.Errorf("Missing or wrong TLV EndTag: got %x, want 0xFE", buf[11]) | ||
| } |
There was a problem hiding this comment.
The TLV validation logic uses several magic numbers (e.g., 16, 0x01, 6, 0xFE). These correspond to values defined in bpf/kmesh/workload/sendmsg.c. To improve readability and consistency, please define these values as constants in the Go test as well. For example:
const (
tlvOrgDstAddrType = 0x01
tlvPayloadType = 0xfe
tlvIPv4DataLen = 6
expectedMinRecvSize = 16
)There was a problem hiding this comment.
Pull Request Overview
This PR introduces unit tests for the sendmsg BPF program, adding test infrastructure to validate TLV (Type-Length-Value) encoding functionality when data is sent through waypoint connections.
Key changes:
- Added sendmsg unit test with mock storage and TLV validation
- Extended BPF test framework to support SkMsg program type
- Added build configuration for the new test file
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/bpf_ut/workload_sendmsg_test.c | Creates mock BPF test environment with storage data for sendmsg testing |
| test/bpf_ut/bpftest/workload_test.go | Implements comprehensive sendmsg test with TLV format validation |
| test/bpf_ut/bpftest/bpf_test.go | Adds SkMsg program type support to test framework |
| test/bpf_ut/Makefile | Adds build target for workload_sendmsg_test.o |
| bpf/kmesh/workload/sendmsg.c | Adds sockhash map definition for socket management |
test/bpf_ut/bpftest/workload_test.go
Outdated
| }) | ||
| startLogReader(coll) | ||
| // Get the sockhash and the program | ||
| sockMap := coll.Maps["km_socket"] |
There was a problem hiding this comment.
The map name "km_socket" doesn't match the map defined in sendmsg.c which is "map_of_kmesh_socket". This will cause the test to fail as sockMap will be nil.
| sockMap := coll.Maps["km_socket"] | |
| sockMap := coll.Maps["map_of_kmesh_socket"] |
| { | ||
| void *storage = NULL; | ||
| storage = bpf_sk_storage_get(map, sk, value, flags); | ||
| if (!storage && map == &map_of_sock_storage) { |
There was a problem hiding this comment.
The map reference "map_of_sock_storage" is not defined or included in this test file, which will cause a compilation error.
| Port: clientPort, | ||
| }, | ||
| Timeout: 2 * time.Second, | ||
| }).Dial("tcp4", serverSocket) |
There was a problem hiding this comment.
The defer statement should be placed after the error check for the Dial operation. If Dial fails, conn will be nil and calling Close() on it will panic.
| }).Dial("tcp4", serverSocket) | |
| }).Dial("tcp4", serverSocket) | |
| if err != nil { | |
| t.Fatal(err) | |
| } |
test/bpf_ut/bpftest/workload_test.go
Outdated
| } | ||
| t.Logf("Server received data: % x", buf[:n]) | ||
| //check | ||
| if n < 16 { |
There was a problem hiding this comment.
The magic number 16 should be defined as a constant or calculated based on the TLV structure (1 byte type + 4 bytes length + 4 bytes IP + 2 bytes port + 1 byte end tag = 12 bytes minimum, not 16).
| if n < 16 { | |
| const MinTLVSize = 12 // 1 byte type + 4 bytes length + 4 bytes IP + 2 bytes port + 1 byte end tag | |
| if n < MinTLVSize { |
|
Run make gen beforn push your commit if you add or change bpf map. |
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
hzxuzhonghu
left a comment
There was a problem hiding this comment.
just find it missed submit comments
bpf/kmesh/workload/sendmsg.c
Outdated
| struct { | ||
| __uint(type, BPF_MAP_TYPE_SOCKHASH); | ||
| __type(key, struct bpf_sock_tuple); | ||
| __type(value, __u32); | ||
| __uint(max_entries, MAP_SIZE_OF_MANAGER); | ||
| __uint(map_flags, 0); | ||
| } map_of_kmesh_socket SEC(".maps"); |
There was a problem hiding this comment.
Yes, why define a map here, we already have one in bpf/kmesh/workload/sockops.c
test/bpf_ut/Makefile
Outdated
| $(QUIET) $(CLANG) $(CLANG_FLAGS) $(TC_FLAGS) -c $< -o $@ | ||
|
|
||
| WORKLOAD_SENDMSG_FLAGS = -I$(ROOT_DIR)/bpf/kmesh/ -I$(ROOT_DIR)/bpf/kmesh/workload/include -I$(ROOT_DIR)/api/v2-c | ||
| workload_sendmsg_test.o: workload_sendmsg_test.c |
There was a problem hiding this comment.
I would suggest we moving these test building to a script and keep makefile simple
There was a problem hiding this comment.
I would suggest we moving these test building to a script and keep makefile simple我建议我们将这些测试构建移动到脚本中并使 makefile 保持简单
Should all the previous include directives be combined into a single bash script?
| } | ||
| localIP := get_local_ipv4(t) | ||
| clientPort := 12345 | ||
| serverPort := 54321 |
There was a problem hiding this comment.
instead of using a fixed port, we can let os randomly choose which free port by calling net.Listen(:0) ?
test/bpf_ut/bpftest/workload_test.go
Outdated
| t.Errorf("Unexpected TLV Port: got %d, want 53", port) | ||
| } | ||
|
|
||
| if buf[11] != 0xFE { |
There was a problem hiding this comment.
can you abstract a utility function to parse tlv protocol? So we can resue it in other tests as well
There was a problem hiding this comment.
I need to verify the IP, port, and the correctness of the format specifically, so I feel it’s not really reusable. Do you think it’s necessary to abstract it?If abstracted, the message, IP, and port need to be passed as parameters. IPv4 and IPv6 should share the logic first when needed, right?
There was a problem hiding this comment.
can you abstract a utility function to parse tlv protocol? So we can resue it in other tests as well你能抽象出一个实用函数来解析 TLV 协议吗?这样我们就可以在其他测试中重复使用它了。
can you abstract a utility function to parse tlv protocol? So we can resue it in other tests as well你能抽象出一个实用函数来解析 TLV 协议吗?这样我们就可以在其他测试中重复使用它了。
Yes, I’ve abstracted the TLV parsing logic into ParseTLVMessage and ValidateTLVMessage in a new file tlv_utils.go, so it can be reused in other tests.
| return storage; | ||
| } | ||
|
|
||
| #define bpf_sk_storage_get mock_bpf_sk_storage_get |
There was a problem hiding this comment.
can you elaborate a little bit how this function can be used
There was a problem hiding this comment.
Okay, I will add some comments.
Codecov Report✅ All modified and coverable lines are covered by tests. Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
| __type(value, __u32); | ||
| __uint(max_entries, MAP_SIZE_OF_MANAGER); | ||
| __uint(map_flags, 0); | ||
| } map_of_kmesh_sendmsg SEC(".maps"); |
There was a problem hiding this comment.
add a comment before this struct to indicate this is used for test only
There was a problem hiding this comment.
@wxnzb is there a way to move this out from this file, i donot want to make it load and attached to kernel when we run kmesh.
hzxuzhonghu
left a comment
There was a problem hiding this comment.
Wait ths map will pollute the dualengine, actually we only need this for test. Is there other way to only load the bpf for test
|
please update |
|
Adding label DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Thanks for the feedback! You're absolutely right about the map pollution concern. I've addressed this by implementing conditional compilation to ensure the map is only compiled during testing: Changes made:
Result:
This approach ensures the BPF map is only loaded during testing, completely addressing the dualengine pollution concern while maintaining full test functionality. |
Signed-off-by: sweeywu <sweetwx6@gmail.com>
Signed-off-by: sweeywu <sweetwx6@gmail.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hzxuzhonghu The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/lgtm |

What type of PR is this?
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?: