Skip to content

[release/1.7] Fix fatal concurrency error in port forwarding#11306

Merged
fuweid merged 1 commit intocontainerd:release/1.7from
ningmingxiao:fix_1.7_panic
Jan 27, 2025
Merged

[release/1.7] Fix fatal concurrency error in port forwarding#11306
fuweid merged 1 commit intocontainerd:release/1.7from
ningmingxiao:fix_1.7_panic

Conversation

@ningmingxiao
Copy link
Contributor

@ningmingxiao ningmingxiao commented Jan 23, 2025

fix:#11302

@k8s-ci-robot
Copy link

Hi @ningmingxiao. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@akhilerm
Copy link
Member

/ok-to-test

Copy link
Member

@austinvazquez austinvazquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me as a stop gap solution; maybe we could also consider updating klog module to pull in JSON encoder changes as a more robust solution but even that is not bullet proof.

Unfortunately I am unable to brainstorm any (unit/integration) test to validate; it is a difficult thing to test. Maybe others have ideas?

If others agree, it would be good to have in release/1.6 as well.

//
// In both cases, we should RESET the httpStream.
klog.ErrorS(err, "forwarding port", "conn", h.conn, "request", p.requestID, "port", portString)
klog.Errorf("forwarding port conn=%p request=%s port=%s failed:%v", h.conn, p.requestID, portString, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit on formatting; non-blocking though.

Suggested change
klog.Errorf("forwarding port conn=%p request=%s port=%s failed:%v", h.conn, p.requestID, portString, err)
klog.Errorf("(conn=%p, request=%s) forwarding port %s failed: %v", h.conn, p.requestID, portString, err)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. do we need update klog to v2.130.1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ningmingxiao; I think the proposed fix here is good enough solution. I did a proof of concept and was able to reproduce the issue using the JSON encoder if the connection has a exported map field that is not annotated to be hidden. See #11302 (comment) for more information.

Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
@fuweid fuweid changed the title fix fatal error: concurrent map iteration and map write [release/1.7] fix fatal error: concurrent map iteration and map write Jan 27, 2025
@fuweid fuweid merged commit e1cdf3c into containerd:release/1.7 Jan 27, 2025
58 checks passed
@ningmingxiao ningmingxiao deleted the fix_1.7_panic branch January 28, 2025 01:46
@dmcgowan dmcgowan changed the title [release/1.7] fix fatal error: concurrent map iteration and map write [release/1.7] Fix fatal concurrency error in port forwarding Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

6 participants