Skip to content

Remove invalid error log when stopping container after containerd restart#11475

Merged
fuweid merged 1 commit intocontainerd:mainfrom
yylt:ctx
Mar 28, 2025
Merged

Remove invalid error log when stopping container after containerd restart#11475
fuweid merged 1 commit intocontainerd:mainfrom
yylt:ctx

Conversation

@yylt
Copy link
Contributor

@yylt yylt commented Mar 4, 2025 •
edited by fuweid
Loading

When containerd restarts and the shim connection is successfully restored, but stopping the container at this time will result in an error log due to using the wrong context

reproduce

1 start pod

crictl runp [pod.json]
crictl create [sandboxid] [container.json] [pod.json]
crictl start [containerid]

2 restart containerd.

3 stop pod by `crictl stop [containerid]

containerd log:

...
ERRO[2025-03-04T16:03:02.137448237+08:00] copy shim log after reload                    error="read /proc/self/fd/10: file already closed" namespace=k8s.io
...

Introduced by #5174

@github-project-automation github-project-automation bot moved this to Needs Triage in Pull Request Review Mar 4, 2025
@k8s-ci-robot
Copy link

k8s-ci-robot commented Mar 4, 2025

Hi @yylt. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@dosubot dosubot bot added the area/runtime Runtime label Mar 4, 2025
@yylt
Copy link
Contributor Author

yylt commented Mar 4, 2025

/cc @fuweid @AkihiroSuda

@yylt yylt force-pushed the ctx branch 2 times, most recently from 6785f02 to c213941 Compare March 5, 2025 03:16
@yylt
Copy link
Contributor Author

yylt commented Mar 5, 2025

This error doesn't seem to be related to the modification?

    default: E0305 03:42:40.287352   20743 remote_runtime.go:179] RemovePodSandbox "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96" from runtime service failed: rpc error: code = Unknown desc = failed to forcibly stop sandbox "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96": failed to stop sandbox "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96": failed to stop sandbox container "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96" in "SANDBOX_UNKNOWN" state: failed to kill pod sandbox container: ttrpc: closed
    default:     sandbox_run_rollback_test.go:161:
    default:         	Error Trace:	/root/go/src/github.com/containerd/containerd/integration/sandbox_run_rollback_test.go:161
    default:         	Error:      	Received unexpected error:
    default:         	            	rpc error: code = Unknown desc = failed to forcibly stop sandbox "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96": failed to stop sandbox "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96": failed to stop sandbox container "3c9b75ee9dca97196047b4d74792972ce692e989549b1a9fd911c6a6c9a16c96" in "SANDBOX_UNKNOWN" state: failed to kill pod sandbox container: ttrpc: closed
...
 default:     --- FAIL: TestRunPodSandboxWithShimDeleteFailure/JustCleanup (0.23s)

@yylt
use shimCtx for fifo copy
4357a76 
Signed-off-by: yylt <yang8518296@163.com>
@yylt yylt mentioned this pull request Mar 7, 2025
Closed
@yylt yylt changed the title use shimCtx for fifoCopy fix error log copy shim log after reload Mar 27, 2025
@yylt
Copy link
Contributor Author

yylt commented Mar 28, 2025

cc @mikebrow @mxpv

mikebrow
mikebrow approved these changes Mar 28, 2025
View reviewed changes
Copy link
Member

@mikebrow mikebrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fuweid pls check.. https://github.com/containerd/containerd/pull/5174/files

@fuweid fuweid changed the title fix error log copy shim log after reload fix: call checkCopyShimLogError(shimCtx) to avoid expected error log flood Mar 28, 2025
@github-project-automation github-project-automation bot moved this from Needs Triage to Review In Progress in Pull Request Review Mar 28, 2025
@fuweid fuweid added this pull request to the merge queue Mar 28, 2025
@fuweid fuweid added cherry-pick/1.7.x Change to be cherry picked to release/1.7 branch cherry-pick/2.0.x Change to be cherry picked to release/2.0 branch labels Mar 28, 2025
Merged via the queue into containerd:main with commit c07fb69 Mar 28, 2025
56 checks passed
@github-project-automation github-project-automation bot moved this from Review In Progress to Done in Pull Request Review Mar 28, 2025
@fuweid
Copy link
Member

fuweid commented Mar 28, 2025

/cherry-pick release/2.0
/cherry-pick release/1.7

@k8s-infra-cherrypick-robot
Copy link

k8s-infra-cherrypick-robot commented Mar 28, 2025

@fuweid: new pull request created: #11620

Details

In response to this:

/cherry-pick release/2.0
/cherry-pick release/1.7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Mar 28, 2025
Merged
@k8s-infra-cherrypick-robot
Copy link

k8s-infra-cherrypick-robot commented Mar 28, 2025

@fuweid: new pull request created: #11621

Details

In response to this:

/cherry-pick release/2.0
/cherry-pick release/1.7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Mar 28, 2025
Merged
@fuweid fuweid added cherry-picked/1.7.x PR commits are cherry-picked into release/1.7 branch cherry-picked/2.0.x PR commits are cherry picked into the release/2.0 branch and removed cherry-pick/1.7.x Change to be cherry picked to release/1.7 branch cherry-pick/2.0.x Change to be cherry picked to release/2.0 branch labels Mar 28, 2025
@akhilerm akhilerm mentioned this pull request Apr 17, 2025
Merged
@dmcgowan dmcgowan changed the title fix: call checkCopyShimLogError(shimCtx) to avoid expected error log flood Remove invalid error log when stopping container after containerd restart Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mikebrow mikebrow mikebrow approved these changes

@fuweid fuweid fuweid approved these changes

@AkihiroSuda AkihiroSuda Awaiting requested review from AkihiroSuda

Assignees

No one assigned

Labels

area/runtime Runtime cherry-picked/1.7.x PR commits are cherry-picked into release/1.7 branch cherry-picked/2.0.x PR commits are cherry picked into the release/2.0 branch needs-ok-to-test size/XS

Projects

Pull Request Review
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@yylt @k8s-ci-robot @fuweid @k8s-infra-cherrypick-robot @mikebrow @dmcgowan