doc: Document compiler configuration for native depends packages #33902

willcl-ark · 2025-11-18T15:23:18Z

Previously one had to read the Makefile (and various *.mk configuration
files) to see how to correctly override CC and CXX when building native
depends packages.

Detail this in README.md to make it clearer.

DrahtBot · 2025-11-18T15:23:25Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33902.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	hebasto, maflcko, ryanofsky

If your review is incorrectly listed, please copy-paste  into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#33593 (guix: Use UCRT runtime for Windows release binaries by hebasto)
#32162 (depends: Switch from multilib to platform-specific toolchains by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

hebasto

Concept ACK. Thanks!

willcl-ark · 2025-11-18T15:27:42Z

Forgot to say, very happy to drop the second commit if wanted.

hebasto · 2025-11-18T15:30:20Z

depends/README.md

+To ensure both native and target use Clang:
+
+```
+make build_CC=clang build_CXX=clang++ host_CC=clang host_CXX=clang++


How is this better than make build_CC=clang build_CXX=clang++ CC=clang CXX=clang++?

They should be exactly equivalent.

I used my version as I had just described build_CC and host_CC and it would seem a little odd (to me) to exemplify with build_CC and CC, although as I re-read it now this example appears in the CC, CXX section.

I think therefore I should set make build_CC=clang build_CXX=clang++ CC=clang CXX=clang++ as the primary example for this section, and perhaps use make build_CC=clang build_CXX=clang++ host_CC=clang host_CXX=clang++ as an "ultimately explicit" final variation?

In most situations where the compiler needs to be specified, users simply run:

make CC=clang CXX=clang++

Only in rare cases (#33902 (comment), #33902 (comment), #33902 (comment)) does this need to be extended to:

make CC=clang CXX=clang++ build_CC=clang build_CXX=clang++

Only in rare cases

I don't think wanting to build without having a GCC installation should be considered rare? Other than the clang-only Nix example linked too above, we are actively trying to make that "normal" for some of our builds, i.e #30206.

They should be exactly equivalent.

Unfortunately this isn't the case. make build_CC=clang build_CXX=clang++ host_CC=clang host_CXX=clang++ doesn't work properly on a machine with no g++, because Boosts CMake will still try and find g++, and fail. Arguably this is also a CMake issue that we should fix/workaround in some way, because there's no reason for Boost to try and find a compiler, when we aren't compiling anything, and only need to copy headers. In this case, you do need to use make build_CC=clang build_CXX=clang++ CC=clang CXX=clang++.

For example, on a Chimera Linux machine:

gmake build_CC=clang build_CXX=clang++ host_CC=clang host_CXX=clang++ -C depends/ -j12 build_TAR=gtar gmake: Entering directory '/bitcoin/depends' Extracting native_capnp... /bitcoin/depends/sources/capnproto-cxx-1.2.0.tar.gz: OK Preprocessing native_capnp... Configuring native_capnp... -- The CXX compiler identification is Clang 21.1.4 -- Detecting CXX compiler ABI info <snip> done Configuring boost... CMake Error at /usr/share/cmake-4.1/Modules/CMakeDetermineCXXCompiler.cmake:47 (message): Could not find compiler set in environment variable CXX: g++. Call Stack (most recent call first): CMakeLists.txt:13 (project) CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage -- Configuring incomplete, errors occurred! gmake: *** [funcs.mk:344: /bitcoin/depends/aarch64-unknown-linux-musl/.boost_stamp_configured] Error 1

Using CC=clang CXX=clang++:

build_CC=clang build_CXX=clang++ CC=clang CXX=clang++ -C depends/ -j12 build_TAR=gtar gmake: Entering directory '/bitcoin/depends' Extracting native_capnp... /bitcoin/depends/sources/capnproto-cxx-1.2.0.tar.gz: OK Preprocessing native_capnp... Configuring native_capnp... -- The CXX compiler identification is Clang 21.1.4 <snip> done Configuring boost... -- The CXX compiler identification is Clang 21.1.4 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/sbin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Boost: using system layout: include, bin, lib/, lib//cmake

OK so having thought about this a bit more, my preference would be to not actually change the docs (as I have them in this PR), but to amend depends/host/default.mk to better respect when host_CC= host_CXX= have been set. I also posit that a setting of host_CC should override CC.

For depends, if I run make build_CC=clang build_CXX=clang++ host_CC=clang host_CXX=clang++ I expect to be able to build all host and native packages.

I also believe that if I run make build_CC=clang build_CXX=clang++ host_CC=clang host_CXX=clang++ CC=notacompiler CXX=notacompiler++, as host/build_CC/CXX have been defined and are more granular, they should be used over CC/CXX.

I think this commit here implements this, but makefile is really not my strongest suit and I could have easily not implemented what I wanted.

What are your thoughts on this approach?

I don't think that the host_$(tool) variables were ever meant to be set by users, and the same applies to build_$(tool).

(this is my understanding of the code as it was originally introduced; however, @theuni knows better)

More recently, it became clear that native packages also need a way to define their toolchains.

As #23571 didn’t gain enough support, I ended up using the "internal" build_$(tool) variables in 551e13a.

maflcko

lgtm

review ACK 17577646f77e20783ffdd9f322f85e96da2265 🌴

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: review ACK 17577646f77e20783ffdd9f322f85e96da2265 🌴
b6tuB1ff+N8xGzUilBsW1CSxIOpk4gK7SE473C8JMS+vg4aU5H4YbP04c0HL3NEDxTZkxKFJBRcJ9RVAQ9t7DQ==

hebasto · 2025-11-19T10:10:51Z

depends/README.md

+Example that may not work as expected (native tools still use GCC):
+
+```
+make CC=clang CXX=clang++


7f17cf6d6164460086afebbb0f921a7af4475bf9:

This example uses Makefile variable CC and CXX, but the section title refers to "Environment Variables". Both approaches work, of course, but the section could be made clearer.

hebasto · 2025-11-19T10:16:49Z

depends/README.md

+
+Setting `CC` and `CXX` environment variables will override target compilers but **NOT** native tool compilers. Native tools will still use the default GCC unless explicitly overridden with `build_*` variables.
+
+Example that may not work as expected (native tools still use GCC):


7f17cf6d6164460086afebbb0f921a7af4475bf9

This comes across as a bit discouraging and confusing. When I say “I build depends with Clang”, I’m referring to the host-specific packages. I’m not concerned about which compiler is used for the native tools, as long as they build without issues.

I’m not concerned about which compiler is used for the native tools, as long as they build without issues.

at least for me the issue was that they did not build at all, because the gcc version was ancient on OpenSuse Leap (or Alma Linux 8, ....)

If using something like a Nix devShell with no gcc available at all, compilation just fails too.

I’m not concerned about which compiler is used for the native tools, as long as they build without issues.

at least for me the issue was that they did not build at all, because the gcc version was ancient on OpenSuse Leap (or Alma Linux 8, ....)

The same issue occurs on NetBSD.

depends/README.md

hebasto · 2025-11-19T11:32:10Z

depends/README.md


 Skip the following packages if you don't intend to use the GUI and will build with [`NO_QT=1`](#dependency-options):

-    apt install bison g++ ninja-build pkgconf python3 xz-utils
+```
+apt install bison g++ ninja-build pkgconf python3 xz-utils


Other native package, not only native_qt, require g++.

I agree, but going to leave that from this PR for now.

hebasto

ACK b1b559f83f629c7d0976c05d07cbc6ec04e84095.

depends/README.md

Previously one had to read the Makefile (and various *.mk configuration files) to see how to correctly override CC and CXX when building native depends packages. Detail this in README.md to make it clearer.

As we are touching the file use modern codeblock syntax throughout.

hebasto

re-ACK 7dd714a.

maflcko · 2025-11-19T16:20:31Z

re-ACK 7dd714a 🔔

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: re-ACK 7dd714ae71fd18eda82ab4b43d4cecc047b87a2d 🔔
GdGC0CV1oagJPrWcSYTtdbnzJVbACdpgtwV3w0kHQgMB3ZEIhZNTp9Mr5d0DHvusR7eUHFQEauWPXPwpa0cUDw==

maflcko · 2025-11-19T16:21:06Z

Could use doc: or build: prefix in title?

fanquake

See #33902 (comment).

willcl-ark · 2025-12-02T10:45:18Z

See #33902 (comment).

As I describe here I have a commit which if host_CC is set, and** CC is unset, will set CC to host_CC (and similarly for CXX and friends).

In my opinion this would result in the expected behaviour:

If {var} is set, use that
If not check if host_{var} is set, and use that
Otherwise, fallback to a default

I don't see a use-case where someone purposefully setting host_{var} would ever expect {var} to take precedence for host buildilng, so don't think having both set would work "unexpectedly"?

@hebasto notes that these variables are apparently not supposed to be set by users though.

If that is the case, perhaps then instead of introducing more opinionated fallback heirarchies we should look at re-working how toolchains are defined more holistically? I don't see an obvious/clean approach not involving two sets of env vars though...

hebasto · 2025-12-02T20:46:12Z

If that is the case, perhaps then instead of introducing more opinionated fallback heirarchies we should look at re-working how toolchains are defined more holistically? I don't see an obvious/clean approach not involving two sets of env vars though...

I agree that the depends build subsystem should accept two sets of variables: one for host/target-specific tools and another for native tools.

The naming for the former is well established (CC, CXX, etc) and it makes no senses to use anything else, as this is what developers, integration workflows and testing frameworks (such as OSS-Fuzz) expect.

For the latter, many projects use the $(tool)_FOR_BUILD pattern, which we might also adopt.

hebasto · 2025-12-04T12:17:26Z

Previously one had to read the Makefile (and various *.mk configuration
files) to see how to correctly override CC and CXX when building native
depends packages.

I also wanted to highlight the distinction between environment variables and Makefile variables. There should be no difference in behaviour between the two when supplied by the user, provided they are recognised by the build system. But that is not currently the case. See: #29963.

ryanofsky

Code review ACK 7dd714a. Nice documentation improvement.

I'm a little confused about hebasto's comments #33902 (comment) and #33902 (comment) which seem to say we should not recommend setting variables like host_CC and host_CXX because setting CC and CXX does the same thing and these are more standard. If so, would it be better for README.md to remove the host_ prefixes? I assume current version is ok given the ACKs, but this seems like something that would be good to follow up on or at least respond to.

Also fanquake's comment #33902 (comment) seems to say that setting CC/CXX works better than host_CC/host_CXX in practice? So maybe the readme should be updated to just drop the host_ prefix entirely if the host_ prefixed variables are less standard and don't work as well as the unprefixed variables.

hebasto · 2025-12-15T13:57:59Z

I'm a little confused about hebasto's comments #33902 (comment) and #33902 (comment) which seem to say we should not recommend setting variables like host_CC and host_CXX because setting CC and CXX does the same thing and these are more standard.

Yes. That's my point.

Also fanquake's comment #33902 (comment) seems to say that setting CC/CXX works better than host_CC/host_CXX in practice? So maybe the readme should be updated to just drop the host_ prefix entirely if the host_ prefixed variables are less standard and don't work as well as the unprefixed variables.

I'll support such changes once someone proposes them.

hebasto reviewed Nov 18, 2025

View reviewed changes

maflcko approved these changes Nov 18, 2025

View reviewed changes

DrahtBot requested a review from hebasto November 18, 2025 15:53

hebasto reviewed Nov 19, 2025

View reviewed changes

DrahtBot requested a review from hebasto November 19, 2025 10:17

hebasto reviewed Nov 19, 2025

View reviewed changes

depends/README.md Outdated Show resolved Hide resolved

DrahtBot requested a review from hebasto November 19, 2025 10:27

willcl-ark force-pushed the doc-depends-cc branch from 1757764 to b1b559f Compare November 19, 2025 11:00

hebasto reviewed Nov 19, 2025

View reviewed changes

hebasto approved these changes Nov 19, 2025

View reviewed changes

depends/README.md Outdated Show resolved Hide resolved

depends/README.md Outdated Show resolved Hide resolved

depends/README.md Outdated Show resolved Hide resolved

DrahtBot requested a review from maflcko November 19, 2025 11:38

willcl-ark added 2 commits November 19, 2025 11:45

doc: Detail compiler configuration for depends

75155c0

Previously one had to read the Makefile (and various *.mk configuration files) to see how to correctly override CC and CXX when building native depends packages. Detail this in README.md to make it clearer.

doc: modernise depends/README.md

7dd714a

As we are touching the file use modern codeblock syntax throughout.

willcl-ark force-pushed the doc-depends-cc branch from b1b559f to 7dd714a Compare November 19, 2025 11:45

DrahtBot added the CI failed label Nov 19, 2025

hebasto approved these changes Nov 19, 2025

View reviewed changes

DrahtBot removed the CI failed label Nov 19, 2025

willcl-ark changed the title ~~Document compiler configuration for native depends packages~~ doc: Document compiler configuration for native depends packages Nov 19, 2025

DrahtBot added the Docs label Nov 19, 2025

fanquake requested changes Nov 21, 2025

View reviewed changes

DrahtBot mentioned this pull request Nov 22, 2025

depends: Switch from multilib to platform-specific toolchains #32162

Open

psam21 mentioned this pull request Nov 25, 2025

depends: Fix native_capnp to respect build_CC and build_CXX #33937

Closed

DrahtBot mentioned this pull request Dec 2, 2025

depends, doc: Add tcl as build dependency for sqlite package #33975

Closed

DrahtBot mentioned this pull request Dec 5, 2025

guix: Use UCRT runtime for Windows release binaries #33593

Open

ryanofsky approved these changes Dec 15, 2025

View reviewed changes


		Setting `CC` and `CXX` environment variables will override target compilers but NOT native tool compilers. Native tools will still use the default GCC unless explicitly overridden with `build_*` variables.

		Example that may not work as expected (native tools still use GCC):

doc: Document compiler configuration for native depends packages #33902

Are you sure you want to change the base?

doc: Document compiler configuration for native depends packages #33902

Conversation

willcl-ark commented Nov 18, 2025

Uh oh!

DrahtBot commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

Uh oh!

hebasto left a comment

Choose a reason for hiding this comment

Uh oh!

willcl-ark commented Nov 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hebasto Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maflcko left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hebasto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hebasto left a comment

Choose a reason for hiding this comment

Uh oh!

maflcko commented Nov 19, 2025

Uh oh!

maflcko commented Nov 19, 2025

Uh oh!

fanquake left a comment

Choose a reason for hiding this comment

Uh oh!

willcl-ark commented Dec 2, 2025

Uh oh!

hebasto commented Dec 2, 2025

Uh oh!

hebasto commented Dec 4, 2025

Uh oh!

ryanofsky left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hebasto commented Dec 15, 2025

Uh oh!

Reviewers

DrahtBot commented Nov 18, 2025 •

edited

Loading

hebasto Nov 19, 2025 •

edited

Loading

ryanofsky left a comment •

edited

Loading