Add default timeout to pkg/plugins/client#25794
Add default timeout to pkg/plugins/client#25794anusha-ragunathan merged 1 commit intomoby:masterfrom
Conversation
|
Signed my patch. |
|
ping @anusha-ragunathan PTAL |
|
gofmt errors:
|
|
@chenchun also ---> Making bundle: validate-lint (in bundles/1.13.0-dev/validate-lint)
Errors from golint:
pkg/plugins/client.go:20:2: const defaultHttpTimeOut should be defaultHTTPTimeOut
Please fix the above errors. You can test via "golint" and commit the result. |
|
@anusha-ragunathan @tophj-ibm Fixed the format errors. |
|
@chenchun : Plugin calls already have a timeout mechanism. It is not at the HTTP Client request level. It is application specific (plugin specific in this case). There's exponential backoff (2, 4, 8, 16 seconds...), with retries, with a maximum timeout of 30 seconds. You were probably waiting in a longer backoff time period (eg 16 seconds) when you experienced the issue. You should see daemon log entries such as https://github.com/docker/docker/blob/master/pkg/plugins/client.go#L135 Given that, I think this PR can be closed. |
|
I don't keep the log. But I believe it is not the backoff retry that was happening. HTTP request timeout is a basic need if the server takes a long time to reply or even not given a reply like the test I write. In this situation the application specific retry won't work. So I don't agree with you. |
|
Both cases you mention should be handled by the current plugin timeout mechanism. |
|
@anusha-ragunathan It happens again and it hangs on callWithRetry for 182 minutes. It is much more than If the http server never reply, the |
|
The full stack-trace helps. You are correct. The application specific backoff/retry logic is for errors that are returned from the http request (eg, connection refused, no socket available to connect, etc). If the call is blocked, then we will not timeout. Reopening, since this PR fixes a valid issue. |
pkg/plugins/client.go
Outdated
There was a problem hiding this comment.
3s is low, especially given that we are more lenient with non-blocking http errors (max of 16s).
go-connections uses 32s for default socket timeouts. https://github.com/docker/docker/blob/3d13fddd2bc4d679f0eaa68b0be877e5a816ad53/vendor/src/github.com/docker/go-connections/sockets/sockets.go#L11 Lets do the same.
There was a problem hiding this comment.
Actually, I'm surprised that the 32s net.Dial timeout didnt kick in.
There was a problem hiding this comment.
It's because the http client is successfully connected to the server. It's the server that didn't response to the http request.
There was a problem hiding this comment.
Changed timeout to 32s
|
LGTM |
|
@chenchun I know it was there before your PR, but if we could add to the timeout constants a |
Signed-off-by: Chun Chen <ramichen@tencent.com>

Happened to find some
docker createcommand hangs for a long time due to remote volume plugin. The goroutine stack is as follows. I believe timeout is necessary.