Skip to content

Reproduce bazel transitions bug, where build settings delta is used to determine output directory hash#124

Closed
amberdixon wants to merge 1 commit intomasterfrom
amber/repro-bazel-transitions-bug
Closed

Reproduce bazel transitions bug, where build settings delta is used to determine output directory hash#124
amberdixon wants to merge 1 commit intomasterfrom
amber/repro-bazel-transitions-bug

Conversation

@amberdixon
Copy link
Copy Markdown
Collaborator

Bazel computes the output directory hash for artifacts under bazel-out when there is a transition. It computes the hash by only looking at the transition build settings which have changed. As a result, the same target built with the same build setting can be placed into different output directories, depending on which target transitioned into it. This makes it impossible to determine where we can expect to find a built artifact.

Here's an example of how to repro this problem from this branch:

# First, build the framework FW. Note that it gets placed in the directory with hash `7bf874b56ea0`
~/Development/rules_ios master * bazelisk build tests/ios/app:FW
INFO: Analyzed target //tests/ios/app:FW (1 packages loaded, 82 targets configured).
INFO: Found 1 target...
Target //tests/ios/app:FW up-to-date:
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/FW
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/Modules/module.modulemap
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/Headers/FW.h
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/Headers/FW-umbrella.h
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/PrivateHeaders/FW_Private.h
INFO: Elapsed time: 3.435s, Critical Path: 3.18s
INFO: 10 processes: 9 darwin-sandbox, 1 local.
INFO: Build completed successfully, 14 total actions

# Then, build the application App that depends on FW
~/Development/rules_ios master * bazelisk build tests/ios/app:App
INFO: Analyzed target //tests/ios/app:App (0 packages loaded, 121 targets configured).
INFO: Found 1 target...
Target //tests/ios/app:App up-to-date:
  bazel-bin/tests/ios/app/App.ipa
INFO: Elapsed time: 4.517s, Critical Path: 4.10s
INFO: 15 processes: 13 darwin-sandbox, 2 local.
INFO: Build completed successfully, 21 total actions

# Notice that there are now two configs for FW. They have also been placed in two different output directories:
~/Development/rules_ios master * bazelisk cquery tests/ios/app:FW
INFO: Analyzed target //tests/ios/app:FW (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
//tests/ios/app:FW (584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4)
//tests/ios/app:FW (660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799)
INFO: Elapsed time: 0.094s
INFO: 0 processes.
INFO: Build completed successfully, 0 total actions

~/Development/rules_ios amber/repro-bazel-transitions-bug ls bazel-out/*/bin/tests/ios/app/FW/FW.framework
bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework:
FW             Headers        Modules        PrivateHeaders

bazel-out/applebin_ios-ios_x86_64-dbg-ST-e763699e1aa1/bin/tests/ios/app/FW/FW.framework:
FW             Headers        Modules        PrivateHeaders

# diffing the bazel configs indicates that the same copy of FW was placed in different output directories.
~/Development/rules_ios amber/repro-bazel-transitions-bug bazelisk config 584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4 > /tmp/build-fw-alone
~/Development/rules_ios amber/repro-bazel-transitions-bug bazelisk config 660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799 > /tmp/build-fw-with-app

~/Development/rules_ios amber/repro-bazel-transitions-bug diff /tmp/build-fw-alone /tmp/build-fw-with-app 
1,2c1,2
< BuildConfiguration 584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4:
< Skyframe Key: BuildConfigurationValue.Key[584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4]
---
> BuildConfiguration 660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799:
> Skyframe Key: BuildConfigurationValue.Key[660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799]
24c24
<   affected by starlark transition: [apple_split_cpu]
---
>   affected by starlark transition: [apple configuration distinguisher, apple_platform_type, cpu, ios_minimum_os]
79c79
<   transition directory name fragment: ST-e763699e1aa1
---
>   transition directory name fragment: ST-7bf874b56ea0

# The only difference is the set of build settings that was changed by the transition. The values of the build settings themselves were the exact same for both builds.

…t when there is a transition. It computes the hash by only looking at the transition build settings which have changed. As a result, the same target built with the same build setting can be placed into different output directories, depending on which target transitioned into it. This makes it impossible to determine where we can expect to find a built artifact.
@amberdixon
Copy link
Copy Markdown
Collaborator Author

THis PR was just for demonstration of a bazel bug, so closing it.

@amberdixon amberdixon closed this Oct 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant