Skip to content

Large output of processes with TTY gets occasionally truncated #3738

@danail-branekov

Description

@danail-branekov

Description

Hi containerd,

We (CF Garden) are experiencing occasional container process output being cut out in our CI environment. We managed to reproduce the issue via a unit test

The test loops 100 times exec'ing a process with TTY enabled in a container which prints numbers from 1 to 1000000 to its stdout. The test checks the final numbers are printed. Depending on the runtime we use to run the test with we get the following results:

  • When running with V1 (io.containerd.runtime.v1.linux) the test would either fail with a process output being cut out, or the process being executed (i.e. seq) just blocks writing to stdout. In the cases when the test fails we get the following error:
--- FAIL: TestContainerExecLargeOutputWithTTY (49.19s)
    container_test.go:320: process output does not end with "999999 1000000" at iteration 66, here are the last 20 characters of the output:

         "999931 999932 999933"
FAIL
  • When running with V2 (io.containerd.runc.v2) the test would always hang with the seq process getting stuck writing to stdout.

We verified that the issue is available on both master and 1.3.0

cc @kieron-pivotal

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions