Dynamic extension build support for macOS and glibc Linux #677

crazywhalecc · 2025-03-24T11:26:54Z

What does this PR do?

cc @DubbleClick .

Checklist before merging

If your PR involves the changes mentioned below and completed the action, please tick the corresponding option.
If a modification is not involved, please skip it directly.

If you modified *.php, run composer cs-fix at local machine.
If it's an extension or dependency update, make sure adding related extensions in src/global/test-extensions.php.
If you changed the behavior of static-php-cli, update docs in ./docs/.
If you updated config/xxx.json content, run bin/spc dev:sort-config xxx.

crazywhalecc · 2025-03-24T15:53:47Z

@DubbleClick I've added full support to build shared extensions.

Support glibc-based linux, macOS
Support all external extensions, not only xdebug, theoretically
Support --build-dev separately

Just need to build with --with-dynamic option:

# Linux
bin/spc-gnu-docker build bcmath --build-cli --with-dynamic=xdebug

# macOS
bin/spc build bcmath --build-cli --with-dynamic=xdebug --no-strip

# build other extensions, like swoole, and swoole.so is portable too.
bin/spc build "" --build-cli --with-dynamic=swoole --no-strip

And result is in buildroot/lib/xxx.so.

src/SPC/store/Downloader.php

config/env.ini

config/source.json

src/SPC/builder/BuilderBase.php

src/SPC/command/BuildPHPCommand.php

src/globals/defines.php

src/SPC/builder/macos/MacOSBuilder.php

src/SPC/builder/Extension.php

src/SPC/builder/extension/swoole.php

src/SPC/command/BuildPHPCommand.php

henderkes · 2025-03-26T17:16:39Z

I think I have an idea on how to go from here. I will see if I can get it implemented that way tomorrow morning.

crazywhalecc · 2025-03-27T01:49:50Z

I think I have an idea on how to go from here. I will see if I can get it implemented that way tomorrow morning.

Almost there. But the only thing I'm not sure is that if we need to rebuild php SAPI. It sounds like we are going to build zero static extension with all shared extension: bin/spc build:shared-ext xdebug --debug and got embed SAPI with no extension statically linked and publish shared so file only?

henderkes · 2025-03-27T02:02:47Z

That's why I would prefer building both static and dynamic versions and simply "skipping" the dynamic extension tests in the resulting executable.

henderkes · 2025-03-27T04:55:33Z

[root@localhost static-php-cli]# SPC_LIBC=glibc CC=/usr/bin/gcc CXX=/usr/bin/g++ AR=/usr/bin/ar LD=/usr/bin/ld.gold bin/
spc build "openssl,zlib,curl" --build-shared="zip,xdebug,swoole" --enable-zts --build-cli
     _        _   _                 _
 ___| |_ __ _| |_(_) ___      _ __ | |__  _ __
/ __| __/ _` | __| |/ __|____| '_ \| '_ \| '_ \
\__ \ || (_| | |_| | (_|_____| |_) | | | | |_) |
|___/\__\__,_|\__|_|\___|    | .__/|_| |_| .__/   v2.5.1
                             |_|         |_|
[04:53:26] [I] Build OS:         Linux (x86_64)
[04:53:26] [I] Build SAPI:       cli, embed
[04:53:26] [I] Extensions (6):   zlib,openssl,curl,*zip,*xdebug,*swoole
[04:53:26] [I] Libraries (11):   pkg-config,zlib,openssl,brotli,nghttp2,libcares,curl,libzip
[04:53:26] [I] Strip Binaries:   yes
[04:53:26] [I] Enable ZTS:       yes
[04:53:26] [I] Build Dev:        yes
[04:53:26] [I] Config File Path: /usr/local/etc/php
[04:53:26] [I] PHP Version:      8.4.5
[04:53:26] [I] Build will start after 2s ...
[04:53:28] [I] enabling swoole without extension swoole-hook-pgsql
[04:53:28] [I] enabling swoole without extension swoole-hook-mysql
[04:53:28] [I] enabling swoole without extension swoole-hook-sqlite
[04:53:28] [I] lib [pkg-config] already built
[04:53:28] [I] lib [zlib] already built
[04:53:28] [I] lib [openssl] already built
[04:53:28] [I] lib [brotli] already built
[04:53:28] [I] lib [nghttp2] already built
[04:53:28] [I] lib [libcares] already built
[04:53:28] [I] lib [curl] already built
[04:53:28] [I] lib [libzip] already built
[04:53:28] [I] patching before-configure for curl checks
[04:53:28] [I] Extension [curl] patched before buildconf
[04:53:28] [I] Entering dir: /opt/static-php-cli/source/php-src
[04:53:28] [I] [EXEC] ./buildconf --force
[04:53:29] [I] Extension [curl] patched before configure
[04:53:29] [I] Entering dir: /opt/static-php-cli/source/php-src
[04:53:29] [I] openssl is using --with-openssl=/opt/static-php-cli/buildroot
[04:53:29] [I] zlib is using --with-zlib
[04:53:29] [I] curl is using --with-curl
[04:53:29] [I] [EXEC] ./configure --prefix= --with-valgrind=no --enable-shared=no --enable-static=yes --disable-all --disable-cgi --disable-phpdbg --with-pic --enable-cli --disable-fpm --enable-embed=static --disable-micro --with-config-file-path=/usr/local/etc/php --with-config-file-scan-dir=/usr/local/etc/php/conf.d --enable-zts --disable-zend-signals --enable-zend-max-execution-timers --with-openssl=/opt/static-php-cli/buildroot --with-zlib --with-curl CFLAGS=' -fPIE' CPPFLAGS='-I/opt/static-php-cli/buildroot/include' LDFLAGS='-L/opt/static-php-cli/buildroot/lib' LIBS='-ldl -lpthread -lm'
[04:53:36] [I] cleaning up
[04:53:36] [I] Entering dir: /opt/static-php-cli/source/php-src
[04:53:36] [I] [EXEC] make clean
[04:53:37] [I] building cli
[04:53:37] [I] Entering dir: /opt/static-php-cli/source/php-src
[04:53:37] [I] [EXEC] sed -i "s|//lib|/lib|g" Makefile
[04:53:37] [I] [EXEC] $SPC_CMD_PREFIX_PHP_MAKE EXTRA_CFLAGS='-g -fstack-protector-strong -fpic -fpie -Os -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fno-ident -fPIE -fPIC' EXTRA_LIBS='/opt/static-php-cli/buildroot/lib/libzip.a /opt/static-php-cli/buildroot/lib/libcurl.a /opt/static-php-cli/buildroot/lib/libcares.a /opt/static-php-cli/buildroot/lib/libnghttp2.a /opt/static-php-cli/buildroot/lib/libbrotlidec.a /opt/static-php-cli/buildroot/lib/libbrotlienc.a /opt/static-php-cli/buildroot/lib/libbrotlicommon.a /opt/static-php-cli/buildroot/lib/libssl.a /opt/static-php-cli/buildroot/lib/libcrypto.a /opt/static-php-cli/buildroot/lib/libz.a  -ldl -lpthread -lm' EXTRA_LDFLAGS_PROGRAM='-all-static -Wl,-O1 -pie' cli
[04:53:56] [I] Entering dir: /opt/static-php-cli/source/php-src/sapi/cli
[04:53:56] [I] [EXEC] strip --strip-all php
[04:53:56] [I] Deploying cli file
[04:53:56] [I] [EXEC] cp '/opt/static-php-cli/source/php-src/sapi/cli/php' '/opt/static-php-cli/buildroot/bin/'
[04:53:56] [I] building embed
[04:53:56] [I] Entering dir: /opt/static-php-cli/source/php-src
[04:53:56] [I] [EXEC] sed -i "s|//lib|/lib|g" Makefile
[04:53:56] [I] [EXEC] make -j12 INSTALL_ROOT=/opt/static-php-cli/buildroot EXTRA_CFLAGS='-g -fstack-protector-strong -fpic -fpie -Os -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fno-ident -fPIE -fPIC' EXTRA_LIBS='/opt/static-php-cli/buildroot/lib/libzip.a /opt/static-php-cli/buildroot/lib/libcurl.a /opt/static-php-cli/buildroot/lib/libcares.a /opt/static-php-cli/buildroot/lib/libnghttp2.a /opt/static-php-cli/buildroot/lib/libbrotlidec.a /opt/static-php-cli/buildroot/lib/libbrotlienc.a /opt/static-php-cli/buildroot/lib/libbrotlicommon.a /opt/static-php-cli/buildroot/lib/libssl.a /opt/static-php-cli/buildroot/lib/libcrypto.a /opt/static-php-cli/buildroot/lib/libz.a  -ldl -lpthread -lm' EXTRA_LDFLAGS_PROGRAM='-all-static -Wl,-O1 -pie' install
[04:54:00] [I] running cli sanity check
[04:54:00] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -r "echo \"hello\";"
[04:54:00] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -r "assert(function_exists('openssl_digest'));assert(openssl_digest('123456', 'md5') === 'e10adc3949ba59abbe56e057f20f883e');//if (file_exists('/etc/ssl/openssl.cnf')) {//    assert(file_get_contents('https://example.com/') "'!'"== false);//}"
[04:54:00] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -r "assert(function_exists('gzcompress'));assert(gzdecode(gzencode('aaa')) === 'aaa');"
[04:54:00] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -r "assert(function_exists('curl_init'));assert(function_exists('curl_setopt'));assert(function_exists('curl_exec'));assert(function_exists('curl_close'));\$curl_version = curl_version();if (stripos(\$curl_version['ssl_version'], 'schannel') "'!'"== false) {    \$curl = curl_init();    curl_setopt(\$curl, CURLOPT_URL, 'https://example.com/');    curl_setopt(\$curl, CURLOPT_RETURNTRANSFER, 1);    curl_setopt(\$curl, CURLOPT_HEADER, 0);    \$data = curl_exec(\$curl);    curl_close(\$curl);    assert(\$data "'!'"== false);}"
[04:54:00] [I] running embed sanity check
[04:54:00] [I] Entering dir: /opt/static-php-cli/source/embed-test
[04:54:00] [I] [EXEC] /usr/bin/gcc -o embed embed.c -fPIC -I/opt/static-php-cli/buildroot/include -I/opt/static-php-cli/buildroot/include/php -I/opt/static-php-cli/buildroot/include/php/main -I/opt/static-php-cli/buildroot/include/php/TSRM -I/opt/static-php-cli/buildroot/include/php/Zend -I/opt/static-php-cli/buildroot/include/php/ext -L/opt/static-php-cli/buildroot/lib -lphp -lc -lzip -lcurl -lcares -lnghttp2 -lbrotlidec -lbrotlienc -lbrotlicommon -lssl -lcrypto -lz -ldl -lpthread -lm
[04:54:00] [I] Entering dir: /opt/static-php-cli/source/embed-test
[04:54:00] [I] [EXEC] ./embed
[04:54:00] [I] Building shared extensions ...
[04:54:00] [I] Building extension [zip] as shared extension (zip.so)
[04:54:00] [I] Entering dir: /opt/static-php-cli/source/php-src/ext/zip
[04:54:00] [I] [EXEC] CFLAGS="-fPIC" /opt/static-php-cli/buildroot/bin/phpize
[04:54:00] [I] [EXEC] CFLAGS="-fPIC" ./configure  --with-php-config=/opt/static-php-cli/buildroot/bin/php-config --enable-shared --disable-static
[04:54:01] [I] [EXEC] CFLAGS="-fPIC" make clean
[04:54:01] [I] [EXEC] CFLAGS="-fPIC" make -j12
[04:54:02] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -d "extension=/opt/static-php-cli/buildroot/lib/zip.so" --ri zip
[04:54:02] [I] Building extension [xdebug] as shared extension (xdebug.so)
[04:54:02] [I] Entering dir: /opt/static-php-cli/source/xdebug
[04:54:02] [I] [EXEC] CFLAGS="-fPIC" /opt/static-php-cli/buildroot/bin/phpize
[04:54:02] [I] [EXEC] CFLAGS="-fPIC" ./configure  --with-php-config=/opt/static-php-cli/buildroot/bin/php-config --enable-shared --disable-static
[04:54:03] [I] [EXEC] CFLAGS="-fPIC" make clean
[04:54:03] [I] [EXEC] CFLAGS="-fPIC" make -j12
[04:54:04] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -d "zend_extension=/opt/static-php-cli/buildroot/lib/xdebug.so" --ri xdebug
[04:54:04] [I] Building extension [swoole] as shared extension (swoole.so)
[04:54:04] [I] Entering dir: /opt/static-php-cli/source/php-src/ext/swoole
[04:54:04] [I] [EXEC] CFLAGS="-fPIC" /opt/static-php-cli/buildroot/bin/phpize
[04:54:04] [I] [EXEC] CFLAGS="-fPIC" ./configure --enable-swoole --enable-swoole-coro-time --disable-thread-context --enable-swoole-curl --enable-openssl --enable-cares --with-nghttp2-dir=/opt/static-php-cli/buildroot --disable-swoole-pgsql --disable-swoole-sqlite --with-php-config=/opt/static-php-cli/buildroot/bin/php-config --enable-shared --disable-static
[04:54:06] [I] [EXEC] CFLAGS="-fPIC" make clean
[04:54:06] [I] [EXEC] CFLAGS="-fPIC" make -j12
[04:54:18] [I] [EXEC] /opt/static-php-cli/buildroot/bin/php -n -d "extension=/opt/static-php-cli/buildroot/lib/swoole.so" --ri swoole
[04:54:18] [I]
[04:54:18] [I]    Build complete, used 52.465 s !
[04:54:18] [I]
[04:54:18] [I] Static php binary path: /opt/static-php-cli/buildroot/bin/php
[04:54:18] [I] Shared extension [zip] path: /opt/static-php-cli/buildroot/lib/zip.so
[04:54:18] [I] Shared extension [xdebug] path: /opt/static-php-cli/buildroot/lib/xdebug.so
[04:54:18] [I] Shared extension [swoole] path: /opt/static-php-cli/buildroot/lib/swoole.so
[04:54:18] [I] License path: /opt/static-php-cli/buildroot/license/

Works very well with building both. zip.so failing to load because it's already loaded does not lead to an error but only a warning, and the extension test will still run through.

henderkes · 2025-03-27T04:56:27Z

We should still figure out why swoole.so is trying to link against libbrotli.so instead of using the static library passed to it, though.

crazywhalecc · 2025-03-30T06:25:23Z

I'll leave this alone for a while. Anyway, if we want to implement it, it will eventually require a separate build command, and the documentation's extension support list will need to be changed.

For independent build command, we can detect whether to build embed and required dependencies to save build time and avoid repeated builds. We can also introduce sanity checks if static ext not included. This way we don't have to argue about whether to build at the same time.

henderkes · 2025-03-30T06:47:47Z

I'm actually quite happy with this as it is right now. Works perfectly well and as expected in all my tests.

henderkes · 2025-03-30T06:49:08Z

We can just leave xdebug in at this time and then later on create separate PR's for other extensions.

resolved

Some prerequisites for refactor

8909b62

crazywhalecc added new feature New feature or request kind/framework Issues related to CLI app framework labels Mar 24, 2025

crazywhalecc changed the base branch from main to feat/xdebug-dynamic March 24, 2025 11:27

crazywhalecc changed the title ~~Dynamic extension build support for macOS and glibc Linux (windows in the future)~~ Dynamic extension build support for macOS and glibc Linux Mar 24, 2025

crazywhalecc self-assigned this Mar 24, 2025

crazywhalecc added 3 commits March 24, 2025 22:39

Add SPC_DOCKER_DEBUG=yes option for docker build

76c353e

Refactor, supports shared extension build now !

aa4d4db

phpstan fix

625a03e

crazywhalecc marked this pull request as ready for review March 24, 2025 15:51

crazywhalecc requested a review from henderkes March 24, 2025 15:51

crazywhalecc commented Mar 24, 2025

View reviewed changes

src/SPC/store/Downloader.php Show resolved Hide resolved

crazywhalecc added 3 commits March 25, 2025 00:05

Support zero extension build result

8cd69b2

sort-config

6dec44b

Remove brotli support for shared build

fc08e5c

henderkes requested changes Mar 25, 2025

View reviewed changes

henderkes reviewed Mar 25, 2025

View reviewed changes

src/SPC/builder/extension/swoole.php Outdated Show resolved Hide resolved

crazywhalecc added 2 commits March 25, 2025 13:29

Change to --enable-shared --disable-static

8459754

Remove dev build target

f371106

henderkes previously requested changes Mar 25, 2025

View reviewed changes

src/SPC/command/BuildPHPCommand.php Outdated Show resolved Hide resolved

crazywhalecc added 5 commits March 26, 2025 12:38

Fix shared extension does not build bug

df06a4b

Fix shared extension does not build bug

ae23b72

Add more exception and log

8cb93bc

Change --with-shared to --build-shared

5564559

Fix phpstan

0beb976

crazywhalecc requested a review from henderkes March 26, 2025 04:41

henderkes added 2 commits March 27, 2025 09:36

build static and shared at the same time

acdec64

unixconfigurearg needs to know if currently building shared or static

48f257f

crazywhalecc marked this pull request as draft March 30, 2025 07:03

henderkes marked this pull request as ready for review March 30, 2025 16:23

henderkes merged commit 7ce1375 into feat/xdebug-dynamic Mar 30, 2025

henderkes deleted the dynamic-ext-refactor branch March 30, 2025 16:23

Uh oh!

Dynamic extension build support for macOS and glibc Linux #677

Dynamic extension build support for macOS and glibc Linux #677

Uh oh!

Conversation

crazywhalecc commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist before merging

Uh oh!

crazywhalecc commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

henderkes commented Mar 26, 2025

Uh oh!

crazywhalecc commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

henderkes commented Mar 27, 2025

Uh oh!

henderkes commented Mar 27, 2025

Uh oh!

henderkes commented Mar 27, 2025

Uh oh!

crazywhalecc commented Mar 30, 2025

Uh oh!

henderkes commented Mar 30, 2025

Uh oh!

henderkes commented Mar 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

crazywhalecc commented Mar 24, 2025 •

edited

Loading

crazywhalecc commented Mar 24, 2025 •

edited

Loading

crazywhalecc commented Mar 27, 2025 •

edited

Loading