Skip to content

tools/test/test-setup.sh fails to collect TEST_UNDECLARED_OUTPUTS_DIR in a minimal remote execution environment #11558

@dws

Description

@dws

Description of the problem / feature request:

Using bazel 3.1.0, I remotely executed a test that created a file in TEST_UNDECLARED_OUTPUTS_DIR. This failed as follows:

external/bazel_tools/tools/test/test-setup.sh: line 389: file: command not found
Could not create "/tmp/worker/shard/operations/49e83fda-5340-46c5-a27d-fa393f7c87ca/bazel-out/k8-fastbuild/testlogs/atg/experimental/user/dws/report/test.outputs/outputs.zip": zip not found or failed

It appears that (a) tools/test/test-setup.sh requires a file(1) command to be present, and that (b) our remote execution environment is minimal enough not to provide one.

The relevant fragment of code in test-setup.sh has portability provisions for stat, even handling the case of it not being there at all:

    386       # stat has different flags for different systems. -c is supported by GNU,
    387       # and -f by BSD (and thus OSX). Try both.
    388       file_size="$(stat -f%z "$undeclared_output" 2>/dev/null || stat -c%s "$undeclared_output" 2>/dev/null || echo "Could not stat $undeclared_output")"
    389       file_type="$(file -L -b --mime-type "$undeclared_output")"
    390

It would be great if it had similar provisions for file(1) and pretty much all the other external commands it relies on.

Feature requests: what underlying problem are you trying to solve with this feature?

I would like to remove obstacles to obtaining additional data from the remote execution of tests using bazel.

If the remote test execution mechanism could work without any system commands being present, that would be excellent.
Barring that, if the test encyclopedia could enumerate what the remote test execution environment must provide in order for bazel's internal scripting to work, that would give us something to work off of in configuring our remote execution environment.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Easy to say: set up a remote test executor that does not make any system commands available to the test runner. Then use it to remotely execute a test that leaves a file in TEST_UNDECLARED_OUTPUTS_DIR.

What operating system are you running Bazel on?

Ubuntu 18.04.

What's the output of bazel info release?

release 3.1.0

Have you found anything relevant by searching the web?

No.

@werkt

Metadata

Metadata

Assignees

Labels

P2We'll consider working on this in future. (Assignee optional)not staleIssues or PRs that are inactive but not considered staleteam-Remote-ExecIssues and PRs for the Execution (Remote) teamtype: bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions