summary

This is an attempt to make parallelization of builds safer by telling bazel what resources an action will consume. By default, Bazel allocates 1 CPU and 250M of RAM, and this makes the current methods of parallelization (i.e. --action_env=CMAKE_BUILD_PARALLEL_LEVEL=16) pretty unpleasant, since bazel won't know and will happily schedule 160 cores worth of actions on a 10-core machine. Although it doesn't solve the whole parallelization problem (see #329) it helps a lot; it reduces the length of our internal build by 30% (by allowing us to fine-tune threads on a per-action basis).

core design

• All of the build system rules now take a resource_size attribute, which can be set to a logical size similar to bazel's other sizing parameters (default, tiny, small, medium, large, enormous).
  • Each size is associated with a default cpu and memory allocation.
  • The defaults can be overridden as settings; bazel run @rules_foreign_cc//foreign_cc/settings will list all the settings in .bazelrc format, along with their default values.
• Setting a non-default size will do two things:
  • A resource_set will be passed to the action, which will give bazel some idea about the load the action will create
  • The various parallelization options will be passed to the underlying build systems: CMAKE_BUILD_PARALLEL_LEVEL, GNUMAKEFLAGS, MESON_NUM_PROCESSES, NINJA_JOBS. All vars are passed to all build systems, since they can delegate to each other (for example, cmake calling ninja or make).
  • Note that the size is part of the cacheable interface, and changing it will pop the cache.
• resource_size="default" (the default) does the same thing as what rfcc/bazel does today (no resource_set, no env var overrides)
  • If a user is currently using action_env (e.g. --action_env=CMAKE_BUILD_PARALLEL_LEVEL=4) or similar:
    • resource_size="default" will do the same thing as before the change
    • setting an explicit resource_size via any means (on the action itself, or via @rules_foreign_cc//foreign_cc/settings:size_default) will override the action_env vars.

other notes

• I have also added resource_size declarations to several of the base requirement tools (make|cmake|ninja|pkgconfig)_tool, which drastically speeds up the CI environment and build iteration speed. (This knocks at least 10 minutes off the cmake_tool build time).
• This also adds a wrapper binary for ninja, because ninja doesn't support environment variables; this is how NINJA_JOBS is implemented. It's c++ instead of bash or python because it needs to be invoked after changing the working directory, and the shim scripts that are used to launch bash or python on windows rely on runfiles, which don't work if you change the wd the way rfcc must.
• My understanding of the bazel scheduler algorithm is:
  • resource_sets tell the scheduler approximately how big an action is
  • resource_sets are hints, they are not enforced by anything
  • the scheduler will try to avoid scheduling too badly over the limit (it does allow some overcommit)
  • the scheduler uses --local_resources to determine the limit; see that for defaults
  • if no actions are running and all remaining actions in the queue are over the limit, then the scheduler will schedule actions one at a time.
• AI disclosure: I have used some AI on this PR, but I stress that I (and not the bot) am ultimately responsible for the code. If you're feeling paranoid, the area with the greatest AI involvement is the ninja_wrapper, where I wrote the unix version and then begged codex to save me from having to write windows code.
• Not implemented in this PR:
  • Any method to declare overcommit; for example, to set the resource_set to 10 cpus but then pass CMAKE_BUILD_PARALLEL_LEVEL=12. This is probably implementable as a setting if someone feels so inclined.
  • None of the underlying build systems accept memory-limiting parameters, which means the resource_set memory size is even more of a suggestion than the cpu size is!