./configure supports using the system or third-party-supplied pkg-config utility to find or override the default location of header and library files when building a few C extensions in the standard library, namely from OpenSSL for the _ssl module, libffi for ctypes, and, new in 3.10, Tcl/Tk for _tkinter (bpo-42603). However, currently for 3.10.0, pkg-config usage is broken for libffi and Tcl/Tk (but not OpenSSL). When running ./configure, there is an unexpected warning that is easily overlooked:

[...]
checking for --with-libs... no
./configure: line 10545: PKG_PROG_PKG_CONFIG: command not found
checking for --with-system-expat... no
[...]

PKG_PROG_PKG_CONFIG is a macro provided by GNU Autotools that is supposed to be in aclocal.m4. Unfortunately, it appears to have been inadvertently deleted in 2fc857a5721a5b42bcb696c9cae1bbcc82a91b17 (PR 25860) probably due to an autoconf version mismatch. The net effect is that the configure variable PKG_CONFIG, the location of the pkg-config utility, is undefined so tests in configure for the location of libffi and of Tcl and Tk do not take into account any pkg-config info and use any default locations (i.e. /usr/include). For most builds, that likely still produces the desired results. But it won't if a builder is trying to override these locations or is building on a platform with different default or is using a third-party package manager and pkg-config to supply libraries.  Note, the _ssl module builds are not affected by this problem as the AX_CHECK_OPENSSL macro in aclocal.m4 does not depend on PKG_PROG_PKG_CONFIG to find pkg-config.

It appears that the problem can be worked around by explicitly setting the PKG_CONFIG build variable when invoking configure, something like:

  ./configure [...] PKG_CONFIG=$(which pkg-config)

But the PR 25860 changes to aclocal.a4 should be carefully reviewed and the pkg-config related deletes restored; there might be other problems, too. This is not the first time we've been caught up by unexpected autoconf changes and, as is clear here, it is too easy for them to go unnoticied. Perhaps we should try to figure out how to reduce those risks.

>  it appears to have been inadvertently deleted in 2fc857a5721a5b42bcb696c9cae1bbcc82a91b17 (PR 25860) probably due to an autoconf version mismatch. 

Since then, I have updated the autotools version to the old one in https://github.com/python/cpython/pull/28151
so this should have been fixed by that.
 What I am missing?

Regarding preventing this, I have added a check so the autotools version doesn't get updated incorrectly:

https://github.com/python/cpython/pull/28152

The updates to aclocal.m4 are generated by the aclocal tool of the GNU build system. That, and other things including autoconf, gets run by autoreconf.  Running autoconf by itself is not sufficient.

https://devguide.python.org/setup/#regenerate-configure

https://www.gnu.org/software/autoconf/manual/autoconf-2.68/html_node/autoreconf-Invocation.html

For regeneration I was using:

https://github.com/tiran/cpython_autoconf

as instructed by Christian.

This runs:

"autoreconf -ivf"

with the correct version of autotools required by the openssl changes

I can confirm that running autoreconf with Christian's version doesn't change the git tree:


❯ docker run -v`pwd`:/src tiran/cpython_autoconf
Rebuilding configure script
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: configure.ac: not using Automake
autoreconf: Leaving directory `.'
Done

3.10 on  3.10 [$]  pyenv 3.9.1 took 6s
❯ git status
On branch 3.10
Your branch is up to date with 'upstream/3.10'.

nothing to commit, working tree clean

Ah, the problem is that pkgconfig is *not* installed in that DOckerfile. I can confirm that installing pkg-config and rerunning works.

Created a PR for this:

https://github.com/pablogsal/cpython_autoconf/pull/1

I think we should move this dockerfile to the CPython repo and update the devguide.

+1, I wasn't aware of the existence of that.

Something seems wrong with the openssl detection when the PKG_PROG_PKG_CONFIG is active :(

Christian, could you please take a look? I would like to get this ready for the release

I have updated the container at https://quay.io/repository/tiran/cpython_autoconf to include pkg-config.

New changeset f146ca36f81075f222aa3a1595042597d96dfad3 by Pablo Galindo Salgado in branch '3.10':
bpo-45350: Rerun autoreconf with the pkg-config macros (GH-28707)
https://github.com/python/cpython/commit/f146ca36f81075f222aa3a1595042597d96dfad3

New changeset a25dcaefb7c4eb0767a112cd31fe0b055f168844 by Pablo Galindo Salgado in branch 'main':
bpo-45350: Rerun autoreconf with the pkg-config macros (GH-28708)
https://github.com/python/cpython/commit/a25dcaefb7c4eb0767a112cd31fe0b055f168844

New changeset 2c47b8080b030ad9cdae96a02fa1e533806eb22a by Pablo Galindo (Pablo Galindo Salgado) in branch '3.10':
bpo-45350: Rerun autoreconf with the pkg-config macros (GH-28707)
https://github.com/python/cpython/commit/2c47b8080b030ad9cdae96a02fa1e533806eb22a