First round of breaking ports:

x64-windows-static, x86-windows, x64-windows

• fontconfig: error in config step:

configure: error: Package requirements (freetype2 >= 21.0.15) were not met:

No package 'freetype2' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables FREETYPE_CFLAGS
and FREETYPE_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

x64-linux

• pangolin: error in build step

/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_monitor_unref'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_enumerate_scan_devices'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_new'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_list_entry_get_name'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_enumerate_get_list_entry'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_list_entry_get_next'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_device_new_from_syspath'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_monitor_receive_device'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_device_get_sysname'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_device_get_action'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_unref'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_monitor_enable_receiving'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_monitor_filter_add_match_subsystem_devtype'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_monitor_get_fd'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_device_get_devnode'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_enumerate_unref'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_enumerate_new'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_enumerate_add_match_property'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_monitor_new_from_netlink'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_device_unref'
/mnt/vcpkg-ci/installed/x64-linux/debug/lib/libuvc.so: undefined reference to `udev_enumerate_add_match_subsystem'
collect2: error: ld returned 1 exit status

x64-osx

• libidn2: error in autoconf step

Can't exec "gtkdocize": No such file or directory at /usr/local/Cellar/autoconf/2.71/share/autoconf/Autom4te/FileUtils.pm line 293.

• libtasn1: error in autoconf step

Can't exec "gtkdocize": No such file or directory at /usr/local/Cellar/autoconf/2.71/share/autoconf/Autom4te/FileUtils.pm line 293.

• x264: error in install step

install: mkdir /Users/vagrant/Data/packages/x264_x64-osx/Users/vagrant/Data/installed: File exists
make: *** [install-cli] Error 71

I will investigae step by step in my personal pipeline.

pangolin

looks like a hidden dependency somewhere

libidn2, libtasn1

requires additional brew package. See #17073 (comment)

fontconfig: error in config step:

Strange since freetype installs pc files.

Strange since freetype installs pc files.

Indeed. A local vcpkg install fontconfig on Ubuntu 18.04 worked as expected, picking the vcpkg freetype via pkgconfig.

"brew install gtk-doc"

Maybe vcpkg_configure_make needs to do a vcpkg_find_acquire_program for autoconfig which in turn proposes

brew install autoconf automake gtk-doc

(On Azure pipelines Ubuntu 18.04, vcpkg proposed to apt-get install gperf on Linux. And there is an gperf port in vcpkg.)

Comparing PKG_CONFIG_PATH in fontconfig config.log and in plain MSYS2 MinGW64 shell, I came to the conclusion that even on Windows, this should use : as separator, which in turn implies unixish directory paths. I use cygpath here, as is done in bootstrap and in some ports.

• linux: pangolin, unrelated but I could help to solve it ([vcpkg baseline][pangolin] Set CMAKE_DISABLE_FIND_PACKAGE_XX to ON #17250).
• x64-windows-static: Download errors, unrelated.
• x86-windows: Download errors, unrelated, plus:

  • paraview:

    • first build stops without at step 5678/8375, shortly after
      LINK : fatal error LNK1201: error writing to program database 'D:\buildtrees\paraview\x86-windows-dbg\bin\vtkPVVTKExtensionsIOImage-pv5.8d.pdb'; check for insufficient disk space, invalid path, or insufficient privilege
    • build restarts in non-parallel mode due to "out-of-memory", but fails quickly again, at 32/2698:
      LINK : fatal error LNK1201: error writing to program database 'D:\buildtrees\paraview\x86-windows-dbg\bin\vtkPVVTKExtensionsIOImage-pv5.8d.pdb'; check for insufficient disk space, invalid path, or insufficient privilege ninja: build stopped: subcommand failed.

    Well, paraview has large fan-in (Qt, gdal) so it might be a side effect of some other package's interplay with pkg-config. Unfortunately I don't have a config log from a previuous build of paraview so I cannot see if there are new features enabled now.
    And it builds with dynamic linkage. I have no way to deal with this locally. I can only ask questions:

  • Does the static build create significantly larger files (in particular debugging information)?

  • Is it somewhat known to be flaky in CI, i.e. might it go away in another run?

  • Is there a way to prevent the pointless seond run?

  • Is there a way to reduce disk space requirements, or improve capacity?

On the download errors:
In my personal CI setup on Azure Pipelines, there is a --download-only stage before the actual build stage, passing the artifacts via cache. This helps with consistency, and it is more polite to the providers of the downloads. (Of course this might be too much for all vcpkg.)

Comparing PKG_CONFIG_PATH in fontconfig config.log and in plain MSYS2 MinGW64 shell, I came to the conclusion that even on Windows, this should use : as separator, which in turn implies unixish directory paths. I use cygpath here, as is done in bootstrap and in some ports.

I have to take this back. Pkg-config documentation and source code say that ; is the separator for Windows, and MSYS2 doesn't change this by patches. However, in MSYS2 bash, PKG_CONFIG_PATH uses :. I assume this works because of the implicit transformations which are applied when calling native windows programs such as /mingw62/bin/pkg-config.exe from MSYS2 programs such as bash. Note the difference:

$ /usr/bin/echo PKG_CONFIG_PATH=$PKG_CONFIG_PATH
PKG_CONFIG_PATH=/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig

$ /mingw64/bin/cmake -E environment | grep PKG_CONFIG_PATH
PKG_CONFIG_PATH=D:\msys32\mingw64\lib\pkgconfig;D:\msys32\mingw64\share\pkgconfig

For the record, pkg-config --help will indicate if it is a native windows program: Only in this case it lists --msvc-syntax.

So I need to go one step back and find the actual cause for the fontconfig issue.

Build errors unrelated to PR.

• gtk:x64-windows: Download error
• libidn2:x64-osx: [libidn2] build failure on osx pipeline #17057
• libtasn1:x64-osx: [libtasn1] build failure in osx pipeline #17098

Please re-run. No longer blocked by libidn2/libtasn1.

New build, new mix of build failures....

x64-osx:libmicrohttpd
Build errors during linking:

Undefined symbols for architecture x86_64:
  "_SecCopyErrorMessageString", referenced from:
      _osstatus_error in libgnutls.a(certs.o)
  "_SecTrustSettingsCopyCertificates", referenced from:
      _add_system_trust in libgnutls.a(certs.o)
  "_SecItemExport", referenced from:
      _add_system_trust in libgnutls.a(certs.o)

and similar.
SecCopyErrorMessageString is part of the Security framework on macOS. How did it become needed now?

The configuration of libmicrohttpd tests for dependencies, but the port doesn't declare any dependencies.

• curl
• gettext
• libgnutls

On macOS, it could pick up files provided either from brew or from a port if it is installed. The mixing of brew and overridden prefix creates volatile configurations.

Changed debug build configuration:

Failed build (debug):

pkg_cv_GNUTLS_CFLAGS=-I/Users/vagrant/Data/installed/x64-osx/debug/lib/pkgconfig/../../../include
pkg_cv_GNUTLS_LIBS='-L/Users/vagrant/Data/installed/x64-osx/debug/lib/pkgconfig/../../lib -lgnutls -lintl -liconv -Wl,-framework -Wl,CoreFoundation -lgmp -lhogweed -lgmp -lnettle -ltasn1 -lidn2 -liconv'

Original build (debug):

pkg_cv_GNUTLS_CFLAGS='-I/Users/vagrant/Data/installed/x64-osx/debug/include -I/Users/vagrant/Data/installed/x64-osx/debug/include/p11-kit-1'
pkg_cv_GNUTLS_LIBS='-L/Users/vagrant/Data/installed/x64-osx/debug/lib -lgnutls -lintl -Wl,-framework -Wl,CoreFoundation -lgmp -lunistring -lhogweed -lgmp -lnettle -ltasn1 -lidn2 -liconv -lunistring -lp11-kit'

Changed release build configuration:

Failed build (release):

pkg_cv_GNUTLS_CFLAGS=-I/Users/vagrant/Data/installed/x64-osx/lib/pkgconfig/../../include
pkg_cv_GNUTLS_LIBS='-L/Users/vagrant/Data/installed/x64-osx/lib/pkgconfig/../../lib -lgnutls -lintl -liconv -Wl,-framework -Wl,CoreFoundation -lgmp -lhogweed -lgmp -lnettle -ltasn1 -lidn2 -liconv'

Build (release) before this change:

pkg_cv_GNUTLS_CFLAGS=-I/Users/vagrant/Data/installed/x64-osx/include/p11-kit-1
pkg_cv_GNUTLS_LIBS='-L/Users/vagrant/Data/installed/x64-osx/lib -lgnutls -lintl -Wl,-framework -Wl,CoreFoundation -lgmp -lunistring -lhogweed -lgmp -lnettle -ltasn1 -lidn2 -liconv -lunistring -lp11-kit'

The transition from /Users/vagrant/Data/installed/x64-osx[/debug] to /Users/vagrant/Data/installed/x64-osx/lib/pkgconfig[/debug/..]/../.. is expected: With this PR, the prefix is really taken from the .pc file, not forced by an external parameter.

Collection of ports that are known to have pc files which contain an invalid prefix (not yet passed through vcpkg_fixup_pkgconfig):

• gdal (broken anyway on Linux, but the root of my current efforts)
• geos (will be added to [geos] Relocatable geos-config; dynamic builds on Unix #17616)
• hdf5 (included in [szip, hdf5] Fix mingw import lib names, control linkage #17941)
• netcdf-c (tbd, depends on hdf5)

@dg0yt I would like to understand what this difference means; or more accurately, what the original thing was trying to fix. Is there a reason it was done this way, and why does it no longer have to be done that way?

@strega-nil-ms: Just merge it. the define-prefix was necessary because there was a time when vcpkg_fixup_pkgconfig was removing the prefix variable which was before it was found how to make it relocatable via pcfiledir. Once upon a time vcpkg_configure_meson also had the same line of code but this was already removed.

@strega-nil-ms You cannot really ask me what the original thing was trying to fix because I didn't add it. This PR is my way of asking that question to the vcpkg maintainers, in the light of the side effects it causes. So I can give a summary of what I took out of the feedback and from additional digging in the repository history:

• The original thing was an early attempt to fix non-relocatable pc files.
  This is no longer needed for proper ports because frequent fixes are automated in vcpkg_fixup_pkgconfig, and maintainers need to take care of additional fixes anyway.
• There might have been an expectation that builds would not silently pick system packages.
  But IMO the gdal situation in linux ([GDAL] On Linux completly broken #9068) can be taken as an indication that maintainers should not rely on this side effect. Apart from that, some developers might actually wish to use system packages at least temporarily.

Ah, okay, thanks y'all!