Avoid different transition output directories when only unrelated command line flags changed #12731
Description
Description of the problem / feature request:
Bazel is currently including user defined build settings from the command line, when calculating hash for transition output directory, even if those settings are not affected by any transition. That result in zero cache hits, despite changing a command line build setting for only a small subset of the actions.
The problem only occurs if also enabling transitions (for other options). Switching flags on command line, without any transitions works fine.
Would it make sense to replace bazels current hashing of all starlark options:
and instead only hash those starlark options that have been affected by any transition, similar to how bazel do it for native options with affectedByStarlarkTransition?
Feature requests: what underlying problem are you trying to solve with this feature?
If automatic configuration trimming becomes reality, we would love to use “bazel test …” with transitions and without command line flags, to test all configurations with a single bazel invocation. But that is not feasible with the current transition scalability and our huge number of user defined build settings.
Instead, as a workaround, we consider setting only a few common options via transitions, such as choosing platform for different target hardware. And use command line flags for most other user defined build settings. But that workaround is not feasible due to this ticket.
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
$ bazel clean
# Rebuild both myA and myB
$ bazel build //test:myFoo --//test:myCmdLineFlag=True
# Only myA should be rebuilt, but unfortunately also myB is rebuilt.
$ bazel build //test:myFoo --//test:myCmdLineFlag=False
test/defs.bzl:
def _transition_impl(settings, attr):
return {"//test:myTransitionFlag": attr.myTransitionFlagValue}
myTransition = transition(
implementation = _transition_impl,
inputs = [],
outputs = ["//test:myTransitionFlag"],
)
def foo_impl(ctx):
return [DefaultInfo(files = depset(ctx.files.dep))]
foo = rule(
implementation = foo_impl,
cfg = myTransition,
attrs = {
"dep": attr.label(),
"myTransitionFlagValue": attr.bool(),
"_whitelist_function_transition": attr.label(
default = '@bazel_tools//tools/whitelists/function_transition_whitelist',
),
})
test/BUILD:
load(
"@bazel_skylib//rules:common_settings.bzl",
"bool_flag",
)
load(":defs.bzl", "foo")
bool_flag(
name = "myTransitionFlag",
build_setting_default = False,
)
bool_flag(
name = "myCmdLineFlag",
build_setting_default = False,
)
config_setting(
name = "myTransitionSetting",
flag_values = {"myTransitionFlag": "True"},
)
config_setting(
name = "myCmdLineSetting",
flag_values = {"myCmdLineFlag": "True"},
)
foo(
name = "myFoo",
myTransitionFlagValue = True,
dep = ":myA",
)
genrule(
name = "myA",
outs = ["myA.out"],
srcs = [":myB"],
cmd = "cat $< > $@; " + select({
"//test:myCmdLineSetting": "echo with myCommandLineSetting >> $@",
"//conditions:default": "echo with default >> $@"}),
)
genrule(
name = "myB",
outs = ["myB.out"],
cmd = select({
"//test:myTransitionSetting": "echo with myTransitiveSetting >> $@",
"//conditions:default": "echo with default >> $@"}),
)
What operating system are you running Bazel on?
Linux
What's the output of bazel info release?
3.7.0
Have you found anything relevant by searching the web?
#12171 is related, but not the same.