Skip to content

compile out gofuzz from prod binaries#92491

Merged
k8s-ci-robot merged 1 commit intokubernetes:masterfrom
BenTheElder:fuzz-only-test
Aug 28, 2020
Merged

compile out gofuzz from prod binaries#92491
k8s-ci-robot merged 1 commit intokubernetes:masterfrom
BenTheElder:fuzz-only-test

Conversation

@BenTheElder
Copy link
Copy Markdown
Member

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind api-change
/kind bug

/kind cleanup

/kind deprecation
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #14460

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

A new `nofuzz` go build tag now disables gofuzz support. Release binaries enable this.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 24, 2020
@k8s-ci-robot k8s-ci-robot requested review from fejta and gmarek June 24, 2020 22:30
@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 24, 2020
@BenTheElder
Copy link
Copy Markdown
Member Author

BenTheElder commented Jun 24, 2020

Here's a quick prototype of eliminating the fuzzer from our binaries, closing the oldest open SIG testing bug in this repo.
/cc @dims @liggitt @thockin

This sort of thing is also unfortunately not available under bazel, we probably care most about this in releases anyhow.
EDIT: and we'll see no regression to those builds, it will continue to be compiled in.

I have not yet audited where else we need to do this, I plan to do that once the approach is cleared.

We do use a similar approach already for the providerless build KEP and dockershim-less building.
However, those tags are not enabled by default. This one is.

We could alternatively have a positive tag, and require our tests enable it, but that's a breaking change for external consumers of types like this. This way the change only affects our binaries and doesn't break bazel, similar to the opt-in "compile out cloud providers" providerless tag.

@BenTheElder
Copy link
Copy Markdown
Member Author

/retest

@liggitt
Copy link
Copy Markdown
Member

liggitt commented Jun 25, 2020

if we want this to stick we need a check that the built binaries don't include gofuzz symbols

Comment thread staging/src/k8s.io/apimachinery/pkg/util/intstr/intstr.go Outdated
@liggitt
Copy link
Copy Markdown
Member

liggitt commented Jun 25, 2020

I'm not very familiar with why gofuzz is problematic to have built into binaries... does it actually cause problems or is this just a representative "should be test-only" dependency? if the latter, do we want fuzz-specific build tags or do we really want a "notest" build tag?

@BenTheElder
Copy link
Copy Markdown
Member Author

BenTheElder commented Jun 25, 2020 via email

@fejta fejta removed their request for review June 25, 2020 20:01
Copy link
Copy Markdown
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason the bug was filed was definitely "we shouldn't compile test code into prod" binaries. How important is that? I don't know. There are other places where the FakeFoo is in the same file or same packages as the Foo, too.

I don't hate the pattern. I'm flummoxed why Go doesn't have this handled by default. The natural approach would be to decorate these as +build testonly but that requires anyone who runs go test to get it right. Given that releases are well-controlled, that seems like the right path, so i concur with your approach.

I like the idea of generalizing this to something like not-for-prod, so maybe +build !for_prod or something?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm used to seeing this RIGHT at the top of the file - I would not think to look for it anywhere else.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack.

@BenTheElder
Copy link
Copy Markdown
Member Author

BenTheElder commented Jun 26, 2020

I like the idea of generalizing this to something like not-for-prod, so maybe +build !for_prod or something

I don't have super strong opinions on this, but I was thinking the only reason anyone else should toggle this is if they're consuming this code from another repo and either want access to the fuzzing behavior, or want to avoid the dependency.

FWIW nofuzz as a name is inspired by gofuzz's //+build gofuzz sample code.
EDIT: with above reasons for it being a negative tag instead, to avoid a breaking change unless you somehow happen to be setting this tag already (???)

I don't hate the pattern. I'm flummoxed why Go doesn't have this handled by default. The natural approach would be to decorate these as +build testonly but that requires anyone who runs go test to get it right. Given that releases are well-controlled, that seems like the right path, so i concur with your approach.

I should have mentioned, I actually looked into this but it's been brought up in go and shot down, repeatedly. golang/go#21360 golang/go#14668

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 1, 2020
@fejta-bot
Copy link
Copy Markdown

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@BenTheElder
Copy link
Copy Markdown
Member Author

/retest

@BenTheElder
Copy link
Copy Markdown
Member Author

/retest

@liggitt
Copy link
Copy Markdown
Member

liggitt commented Jul 15, 2020

this looks reasonable to me

@BenTheElder
Copy link
Copy Markdown
Member Author

/priority backlog
will circle back to this in the 1.20 timeframe

@k8s-ci-robot k8s-ci-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jul 21, 2020
@dims
Copy link
Copy Markdown
Member

dims commented Jul 27, 2020

/assign @thockin
/uncc

@k8s-ci-robot k8s-ci-robot removed the request for review from dims July 27, 2020 10:56
Copy link
Copy Markdown
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't know why Go doesn't have a test build-tag built in.
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 27, 2020
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 27, 2020
@justaugustus justaugustus added this to the v1.20 milestone Aug 27, 2020
@k8s-ci-robot k8s-ci-robot merged commit aa084d4 into kubernetes:master Aug 28, 2020
@BenTheElder BenTheElder deleted the fuzz-only-test branch July 9, 2021 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/backlog Higher priority than priority/awaiting-more-evidence. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fuzz() method on IntOrString causes gofuzz linked into prod code

7 participants