deprecate pkg/stdcopy, move to api/stdcopy by thaJeztah · Pull Request #50462 · moby/moby

thaJeztah · 2025-07-21T09:02:02Z

relates to Create separate go modules for api, client, and daemon #49873

The stdcopy package is used to produce and read multiplexed streams for "attach" and "logs". It is used both by the API server (to produce), and the client (to read / de-multiplex).

Move it to the api package, so that it can be included in the api module.

- Human readable description for the release notes

Go-SDK: deprecate `pkg/stdcopy`, which was moved to `api/stdcopy`.

- A picture of a cute animal (not mandatory but encouraged)

thaJeztah · 2025-07-21T09:04:09Z

pkg/stdcopy/stdcopy_deprecated.go

+import (
+	"io"
+
+	"github.com/docker/docker/api/stdcopy"


I was on the fence on the best location for this; it's not a "type" (really?), so it didn't feel right to put it under types/. Also considered if we needed a utils/ or common/ package underneath api/.

☝️ input welcome!

Maybe api/internal/stdcopy, or api/pkg/stdcopy if internalization is too strict

The tricky bit is that it's used by the client; it's the implementation to decode these multiplexed streams, so it can't be internal.

It's basically the implementation to both produce and handle this format;

moby/api/swagger.yaml

Lines 8657 to 8700 in a1ee566

### Stream format

When the TTY setting is disabled in [`POST /containers/create`](#operation/ContainerCreate),

the HTTP Content-Type header is set to application/vnd.docker.multiplexed-stream

and the stream over the hijacked connected is multiplexed to separate out

`stdout` and `stderr`. The stream consists of a series of frames, each

containing a header and a payload.

The header contains the information which the stream writes (`stdout` or

`stderr`). It also contains the size of the associated frame encoded in

the last four bytes (`uint32`).

It is encoded on the first eight bytes like this:

```go

header := [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}

```

`STREAM_TYPE` can be:

- 0: `stdin` (is written on `stdout`)

- 1: `stdout`

- 2: `stderr`

`SIZE1, SIZE2, SIZE3, SIZE4` are the four bytes of the `uint32` size

encoded as big endian.

Following the header is the payload, which is the specified number of

bytes of `STREAM_TYPE`.

The simplest way to implement this protocol is the following:

1. Read 8 bytes.

2. Choose `stdout` or `stderr` depending on the first byte.

3. Extract the frame size from the last four bytes.

4. Read the extracted size and output it on the correct output.

5. Goto 1.

### Stream format when using a TTY

When the TTY setting is enabled in [`POST /containers/create`](#operation/ContainerCreate),

the stream is not multiplexed. The data exchanged over the hijacked

connection is simply the raw data from the process PTY and client's

`stdin`.

thaJeztah · 2025-07-21T09:24:53Z

Derp; need to adjust the validation;

These files import internal code: (either directly or indirectly)
 - pkg/stdcopy/stdcopy_deprecated.go imports github.com/docker/docker/api/stdcopy

The stdcopy package is used to produce and read multiplexed streams for "attach" and "logs". It is used both by the API server (to produce), and the client (to read / de-multiplex). Move it to the api package, so that it can be included in the api module. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah added this to the 29.0.0 milestone Jul 21, 2025

thaJeztah added status/2-code-review impact/deprecation area/go-sdk impact/go-sdk Noteworthy (compatibility changes) in the Go SDK release-blocker PRs we want to block a release on labels Jul 21, 2025

thaJeztah added this to go modules transition Jul 21, 2025

thaJeztah commented Jul 21, 2025

View reviewed changes

thaJeztah force-pushed the move_stdcopy branch from a44464c to e3087f4 Compare July 21, 2025 09:41

thaJeztah requested a review from tianon as a code owner July 21, 2025 09:41

thaJeztah mentioned this pull request Jul 21, 2025

Move remaining image packages to daemon #50446

Merged

vvoland approved these changes Jul 21, 2025

View reviewed changes

thaJeztah requested a review from dmcgowan July 21, 2025 17:03

thaJeztah force-pushed the move_stdcopy branch from e3087f4 to 20d594f Compare July 21, 2025 19:41

dmcgowan approved these changes Jul 21, 2025

View reviewed changes

thaJeztah merged commit 81caaba into moby:master Jul 21, 2025
278 of 281 checks passed

github-project-automation bot moved this to Done in go modules transition Jul 21, 2025

thaJeztah deleted the move_stdcopy branch July 21, 2025 20:50

thaJeztah mentioned this pull request Jul 22, 2025

hack/validate: remove obsolete exception, fix missing alias for stdcopy.Systemerr #50483

Merged

This was referenced Aug 4, 2025

Prepare release notes for v1.52.0-alpha.0 #50625

Merged

Prepare release notes for client/v0.1.0-alpha.0 #50637

Merged

austinvazquez mentioned this pull request Aug 5, 2025

Bump api version in client and main module to v1.52.0-alpha.1 #50641

Merged

dmcgowan mentioned this pull request Aug 6, 2025

Prepare release notes for v2.0.0-alpha.0 #50651

Closed

This was referenced Sep 5, 2025

Prepare release notes for api/v1.52.0-beta.0 #50904

Merged

Prepare release notes for client/v0.1.0-beta.0 #50913

Merged

Prepare release notes for v2.0.0-beta.0 #50918

Merged

dmcgowan mentioned this pull request Sep 24, 2025

Prepare release notes for api/v1.52.0-rc.0 #51030

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deprecate pkg/stdcopy, move to api/stdcopy#50462

deprecate pkg/stdcopy, move to api/stdcopy#50462
thaJeztah merged 1 commit intomoby:masterfrom
thaJeztah:move_stdcopy

thaJeztah commented Jul 21, 2025 •

edited by vvoland

Loading

Uh oh!

thaJeztah Jul 21, 2025

Uh oh!

AkihiroSuda Jul 21, 2025

Uh oh!

thaJeztah Jul 21, 2025

Uh oh!

thaJeztah Jul 21, 2025

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	### Stream format

	When the TTY setting is disabled in [`POST /containers/create`](#operation/ContainerCreate),
	the HTTP Content-Type header is set to application/vnd.docker.multiplexed-stream
	and the stream over the hijacked connected is multiplexed to separate out
	`stdout` and `stderr`. The stream consists of a series of frames, each
	containing a header and a payload.

	The header contains the information which the stream writes (`stdout` or
	`stderr`). It also contains the size of the associated frame encoded in
	the last four bytes (`uint32`).

	It is encoded on the first eight bytes like this:

	```go
	header := [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}
	```

	`STREAM_TYPE` can be:

	- 0: `stdin` (is written on `stdout`)
	- 1: `stdout`
	- 2: `stderr`

	`SIZE1, SIZE2, SIZE3, SIZE4` are the four bytes of the `uint32` size
	encoded as big endian.

	Following the header is the payload, which is the specified number of
	bytes of `STREAM_TYPE`.

	The simplest way to implement this protocol is the following:

	1. Read 8 bytes.
	2. Choose `stdout` or `stderr` depending on the first byte.
	3. Extract the frame size from the last four bytes.
	4. Read the extracted size and output it on the correct output.
	5. Goto 1.

	### Stream format when using a TTY

	When the TTY setting is enabled in [`POST /containers/create`](#operation/ContainerCreate),
	the stream is not multiplexed. The data exchanged over the hijacked
	connection is simply the raw data from the process PTY and client's
	`stdin`.

Conversation

thaJeztah commented Jul 21, 2025 • edited by vvoland Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

AkihiroSuda Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thaJeztah commented Jul 21, 2025 •

edited by vvoland

Loading