Skip to content

Fixed the Istio docker build failure#173

Closed
seflerZ wants to merge 8 commits intoistio:masterfrom
seflerZ:docker-build-fixup
Closed

Fixed the Istio docker build failure#173
seflerZ wants to merge 8 commits intoistio:masterfrom
seflerZ:docker-build-fixup

Conversation

@seflerZ
Copy link
Copy Markdown

@seflerZ seflerZ commented Dec 26, 2019

This PR includes:
✔ Fixed corrupted docker building procedure in Istio component.
✔ Enable "go test" options by setting environment variable $T which is already existed in "Makefile.core.mk"

I'll explain this in details:
👉 I've made the "current directory" mapped to "/work/src/istio.io/istio" because some legacy tests in Istio need path "/src/istio.io/istio" to find the source root. This doesn't affect the final outputs and behaves exactly like "building with local tool chain".
👉 Added the missing "GOBIN" environment variable (Otherwise it will become "/gobin" in building process which will cause test failure).

@seflerZ seflerZ requested a review from a team as a code owner December 26, 2019 09:53
@googlebot googlebot added the cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. label Dec 26, 2019
@istio-testing
Copy link
Copy Markdown
Contributor

Hi @seflerZ. Thanks for your PR.

I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@istio-testing istio-testing added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Dec 26, 2019
@istio-testing istio-testing added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 26, 2019
@seflerZ
Copy link
Copy Markdown
Author

seflerZ commented Jan 14, 2020

@sdake Would you please take a look?

@sdake
Copy link
Copy Markdown
Member

sdake commented Jan 19, 2020

@seflerZ - I have been on PTO for a few weeks - then had flu for a few weeks. Apologies for the long lag in the review. I will take a look right now.

Cheers
-steve

@howardjohn
Copy link
Copy Markdown
Member

One potential benefit of this is you can do things like reference other repos (for example, in go.mod replace istio.io/api => ../api). However, it might be slow to mount everything (or it might not - i have no clue)

-e TARGET_OUT_LINUX="$(TARGET_OUT_LINUX)" \
-e USER="${USER}" \
-e T="${T}" \
-e GOBIN="/work/src/istio.io/istio/bin" \
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not obvious. The environment variable GOBIN is explicitly set in the Dockerfile environment. We do not desire to override it in the Makefile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this needs to apply to all repos

Copy link
Copy Markdown
Author

@seflerZ seflerZ Jan 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I knew the default value for GOBIN is "/gobin". However, this will cause the following problem:

--- FAIL: TestEnvoy (0.56s)
    xds_test.go:116: Failed to setup test: fork/exec /gobin/envoy: no such file or directory

When using local build chain, you can see we explicitly set the "GOBIN" to "GOPATH/bin"

$(info Building with your local toolchain.)
export GOBIN ?= $(GOPATH)/bin
include Makefile.core.mk

endif

The reason is we don't mount "/gobin" as volume in docker container. We run command "make pilot-test" as a fresh container which has no "/gobin/evnoy" in it. Even if we've just generated it with command "make build".

I passed in the GOBIN as argument for generating Envoy binary within the mounted "/work/src/istio.io/istio" directory. So if we changed the default value of GOBIN to "/go/bin", it's also Okay.

export BUILD_WITH_CONTAINER ?= 0

# This is the "go test" options. Default empty value.
export T ?=
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy to permit go test overrides.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW #172 allows arbitrary env

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@howardjohn This is "go test" options, not env. Have you planed to cover it in #172 ? If it is, I'll close this PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its an environment variable that we read and pass to go test, so it should be covered in #172

Copy link
Copy Markdown
Author

@seflerZ seflerZ Jan 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@howardjohn Thank you. I'll close this PR.

ifeq ($(BUILD_WITH_CONTAINER),1)
export TARGET_OUT = /work/out/$(TARGET_OS)_$(TARGET_ARCH)
export TARGET_OUT_LINUX = /work/out/linux_amd64
export TARGET_OUT = /work/src/istio.io/istio/out/$(TARGET_OS)_$(TARGET_ARCH)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using src/istio.io/istio doesn't feel like the correct solution. The correct solution would take into account the environment in the test cases and expect binaries or test files in the correct location.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it is because some legacy unit tests within the Pilot tried to resolve the source root path by matching "/src/istio.io/istio". If we don't intend to do modification here, we have to change the unit tests within Pilot.

This may be a little hard for we don't have any specified pattern to match. For example, if we rest in "[arbitrary path]/pilot/src/core/[here]", how can we tell the root source path since we have replaced "istio" directory name to "work"? If we choice to math "work", what if someone else mapped the "istio" to "mywork" like "mywork/pilot/src/..."?

--mount type=volume,source=gocache,destination="/gocache" \
$(CONDITIONAL_HOST_MOUNTS) \
-w /work $(IMG)
-w /work/src/istio.io/istio $(IMG)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my knee-jerk reaction to this line of change as well as the other src/istio.io/istio changes is they are not the correct approach. I do not disagree that current testing is busted. I think the correct solution to solving the testing problems within a build container involve reading the target environment variable (or as it may be transposed in the istio/istio/Makefile.core.mk file... to a different variable) - to determine the correct testing behavior rather than adding hardcodes. I will give it some more thought. I am avialble on slack for further conversation.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fully understand. But the reality is even you choose to build with your local tool chain, you can't pass unit tests if your source code don't resident in "src/istio.io/istio"(Last time I made a soft link to "src/istio.io/istio" with "istio" and the tests failed). So we all agreed that it is the test code's responsibility to figure out the right path no matter where they have been mapped, but we need time to do refactoring. The problem is, if we don't make changes, we can't event make further testing and deployment.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seflerZ we are headed towards making BUILD_WITH_CONTAINER=1 the default within istio/istio. Anything that detracts from that objective IMO is not ideal. The fact that the test cases need refactoring don't have much if any bearing - other than the work needs to be done, even if it needs to be forced.

@sdake
Copy link
Copy Markdown
Member

sdake commented Jan 19, 2020

One potential benefit of this is you can do things like reference other repos (for example, in go.mod replace istio.io/api => ../api). However, it might be slow to mount everything (or it might not - i have no clue)

bind mounting is cheap resource-wise.

-e TARGET_OUT_LINUX="$(TARGET_OUT_LINUX)" \
-e USER="${USER}" \
-e T="${T}" \
-e GOBIN="/work/src/istio.io/istio/bin" \
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I like this as it may cause changes in the local bin directory. And which envoy would be copied into bin, the local OS (MacOS in my case) or the linux version since the container is using linux? I believe there is another issue which I put a change in the PR comments #19937 for where the linux envoy is polluting the darwin directory structure when building with the container, and this could pollute higher.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ericvn in all cases where BUILD_WITH_CONTAINER=1, GOBIN should contain Linux binaries.

@seflerZ seflerZ closed this Jan 27, 2020
@seflerZ seflerZ deleted the docker-build-fixup branch January 27, 2020 11:42
fjglira pushed a commit to fjglira/istio-common-files that referenced this pull request Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. needs-ok-to-test size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants