feat(extensions): add the PHP extension generator #1649

alexandre-daubois · 2025-06-12T12:56:47Z

After a few weeks working on this with Kevin, I'm happy to propose the extension generator to the review. This PR targets the ext branch and not main, so we can iterate on it before actually shipping the generator in main.

PHP Extension Generator

It's been hinted here and there that FrankenPHP allows to "easily" create PHP extensions written in Go. However, having to understand PHP internal mechanisms can be quite hard and discouraging when it comes to creating an extension. This tool was designed to remove this barrier and make it easier to create an extension in Go and abstract C code and type juggling for most basic use cases.

It looks like Symfony's maker bundle: great to kickstart things and the majority of use cases, help you to greatly reduce the boilerplate, but it cannot cover everything if you need extra optimization and super advanced use cases.

The generator cannot cover all PHP features, but we're doing our best to propose the most used ones.

Note

The documentation is being written and will be proposed in a separate PR. This one's already big enough 🙂

Writing the extension in Go

Thanks to Go directive, we can declare functions, constants, classes and methods right from the Go code. Let's say you have the following code:

package main

import (
	"C"
	"github.com/dunglas/frankenphp"
	"strings"
	"unsafe"
)

// export_php:const
const GLOBAL_CONST = iota

// export_php:classconst MySuperClass
const STR_REVERSE = iota

// export_php:classconst MySuperClass
const STR_NORMAL = iota

// export_php:class MySuperClass
type MyClass struct {
	Name     string
}

// export_php:method MySuperClass::setName(string $name): void
func (mc *MyClass) SetName(v *C.zend_string) {
	mc.Name = frankenphp.GoString(unsafe.Pointer(v))
}

// export_php:method MySuperClass::getName(): string
func (mc *MyClass) GetName() unsafe.Pointer {
	return frankenphp.PHPString(mc.Name, false)
}

// export_php:method MySuperClass::repeatName(int $count, ?int $mode): void
func (mc *MyClass) RepeatName(count int64, mode *int64) {
	str := mc.Name

	result := strings.Repeat(str, int(count))
	if mode != nil && *mode == STR_REVERSE {
		runes := []rune(result)
		for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
			runes[i], runes[j] = runes[j], runes[i]
		}
		result = string(runes)
	}

	if mode == nil || *mode == STR_NORMAL {
		// no-op, just to use the constant :)
	}

	mc.Name = result
}

// export_php:function is_even(int $a): bool
func is_even(a int64) bool {
	return a%2 == 0
}

// export_php:function float_div(float $a, float $b): float
func float_div(a float64, b float64) float64 {
	return a / b
}

Many things to note here:

The use of //export_php:function, //export_php:const, //export_php:classconst, //export_php:class and //export_php:method to export symbols to PHP in our extension
//export_php:function and //export_php:method takes an "argument": the function prototype, which will be directly used by PHP stubs to generate the arginfo file
Support for scalar types in parameters and return types. Note that nullable parameters use pointers, so you can easily check if the parameter is actually null (see RepeatName)
Constants can be global or class-scoped
Type juggling between C and Go is eased thanks to some helper methods (see getName() and setName())
Constants are using iota as their value: their value will be automatically generated when running the generator. You can of course provide a value yourself and it supports floats, bools, strings and integers using binary, octal, decimal and hexadecimal notation.

Type Juggling

Type juggling is a big subject, maybe the hardest part of writing extensions in Go. There's nothing to do with types like int, float and bool. But for example with strings, this is not the same thing. Strings have a special memory representation in the Zend Engine and need conversion before being used in your extension. FrankenPHP provides multiple functions to deal with this, see types.go.

For this particular reason, arrays and objects are not yet supported by the extension generator. It will eventually come but in a second time. Again, the PR is already pretty big. 🙂

What does this generate?

The generator creates 6 files for a given Go file:

ext.c, containing the actual C code of the extension that will call Go for you
ext.go, your actual Go code, with some tweaks in order for everything to work smoothly together with the C code
ext.h, with your module prototype and your constant values if you defined some
ext.stub.php, holding all classes and functions prototypes and definitions. This is used by a PHP tool (provided in php-src) to generate the arginfo file
ext_arginfo.h, generated by the PHP tool that has the C code to declare everything in the Zend Engine (class entries, function entries, etc.)
README.md, a simple markdown file containing all exported symbols of your extension

Opaque Classes

Exported classes are opaque, which means you cannot access properties from your PHP code and you must use exported methods to interact with them. This is done because of technical limitations. Instead of rewriting all PHP rules (type checks and so on) in class hooks, let's make classes opaque. This also ensure that userland cannot corrupt internal state of our object. Note that this is something often done in extensions and a lot of internal PHP classes are opaque: \Directory, \CurlHandle, etc.

Validation

The generator does it's best to give insightful warnings when something is potentially wrong in the Go file containing the extension. For example, it validates that parameter and return types are somehow compatible between the PHP signature and the Go signature and can display messages likes this:

Warning: Go method signature mismatch for 'MySuperClass::hasName': return type mismatch: PHP 'bool' requires Go return type 'bool' but found 'string'
Warning: Go method signature mismatch for 'MySuperClass::repeatName': return type mismatch: PHP 'string' requires Go return type 'unsafe.Pointer' but found ''

Compiling The Extension

After running the generator on a Go file, a build directory is created next to the file. You just need to include this directory in the FrankenPHP compilation step using xcaddy --with-module (or equivalent), and you're done! The above example would allow to write such:

$class = new MySuperClass();
$class->setName('A name !');

$class->repeatName(4, MySuperClass::STR_REVERSE);

var_dump($class->getName()); // outputs `string(32) "! eman A! eman A! eman A! eman A"`

That's it: you created an extension only writing Go and not a line of C!

alexandre-daubois · 2025-06-13T09:25:03Z

Just added more validation on parameters and return types, see the Validation section in the PR description.

withinboredom

Looking forward to taking this for a test drive!

withinboredom · 2025-06-13T11:01:22Z

internal/extgen/classparser.go

+	typeMap := map[string]string{
+		"string": "string",
+		"int":    "int", "int64": "int", "int32": "int", "int16": "int", "int8": "int",
+		"uint": "int", "uint64": "int", "uint32": "int", "uint16": "int", "uint8": "int",
+		"float64": "float", "float32": "float",
+		"bool": "bool",
+	}


do we need to handle other "weird" php types, like callable? For example, we had an onRequestFailure($closure) extension to handle some logging on failed requests.

Ultimately it would be nice to support all types yes! I need to do more investigation on how this could work with Go, thus the restricted types for now. But definitely, if we can support all types, that would be super nice

IIRC (I no longer work there), it was just stored as a pointer to the function info + the other function thing (I can't remember the name of it) that needs to be kept. On the go side, we just called a helper passing those (think something like ExecutePHPCallback(funcStruct)) that would then do a bunch of monkeying around in C to ensure it got dispatched on the correct thread. In this case, you could probably just pass the pointers via a channel waiting on the appropriate thread. Our case violated the C-Go-C rules (same as the old fibers issue), so -- figuring out how to get around that will be interesting, I'm sure.

henderkes · 2025-06-13T14:28:21Z

This is a really cool idea, but I'm wondering how suitable Go really is to develop php extensions. The runtime will limit this to static compilation (unless doing some voodoo stuff with -buildmode=c-shared, linking against a shared Go runtime, loading the library in via php's extension loading mechanism again), which means users will have to be actively involved in how to compile frankenphp & extensions. And C++'s memory model and C interoperability are much better than Go's, as is performance for crucial tasks.

This will essentially make for a rather narrow range of application: developers with experience in Go, that wish to tightly couple their application(s) to FrankenPHP, who have no experience with zend and C/C++. Or you really need the functionality of an existing Go library. That being said, I'm excited to see this being developed and will be happy if it eventually takes off. Are you going for a small demonstration on Tuesday?

withinboredom · 2025-06-14T01:06:47Z

but I'm wondering how suitable Go really is to develop php extensions.

Quite suitable actually! Many PHP devs know Go and even use Go. Go and PHP go quite well together; IMHO. In a lot of ways, Go reminds me of PHP in the early days -- back when people knew PHP, not frameworks.

That being said,

The runtime will limit this to static compilation

There are Go plugin libraries. Caddy even looked into using them at one point in the early days (I remember having a discussion on Twitter with @mholt way back then). They're much more mature today, so it wouldn't be all that complicated to add a layer onto this that allows you to use PHP extensions as a plugin via a frankenphp extension. I have no idea what the performance impact would be, but I doubt it would be too insane.

henderkes · 2025-06-15T07:19:51Z

Quite suitable actually! Many PHP devs know Go and even use Go. Go and PHP go quite well together; IMHO. In a lot of ways, Go reminds me of PHP in the early days -- back when people knew PHP, not frameworks.

I didn't really mean from a programming language point of view, but runtime and memory model complexities. It's typically much easier to extend systems in a language that closely resembles the language used to create the system. That's why we use C++ to interface with games written in C++ (using the same compiler toolchain), and C# to interface with games written in C#. There are already some troubles with type conversions now, I'm not sure how well it'll go with callables, array types (how much overhead will the mapping introduce?) and structs.

I have no idea what the performance impact would be, but I doubt it would be too insane.

At the very least it adds two layers of indirection with every function call. On top of that, from what I understand, longer function calls may also cause a context switch. And last but not least, while Go is fast, it's not quite as fast as using C/C++ directly. It has great benefits too, especially in cases involving concurrency and the ease of using libraries, so there will definitely be a field where it makes sense, but I have the feeling that native C extensions will likely be the better choice in a lot of scenarios.

alexandre-daubois · 2025-06-15T08:47:35Z

Indeed, you've just pointed out something important! We're well aware that the development of highly optimized extensions with particular performance needs will always require the use of C, at one time or another.
The uses we had in mind when discussing this novelty are indeed: the possibility of using goroutines (we can imagine a worker like Symfony Messenger, directly integrated into PHP!) but also the wrapping of libs that are, among others, the success of Go.

So yes, when you mention concurrency and Go libraries, your right on target!

withinboredom · 2025-06-16T16:23:32Z

@alexandre-daubois is there a draft of the documentation? I'd love to write a test extension, but I'm not sure how to get started.

dunglas · 2025-06-16T16:26:05Z

I created one. It will be published tomorrow during JetBrains PHPVerse!

AlliBalliBaba · 2025-06-16T19:23:47Z

Looks like a ton of effort went into this extension generator, also excited to try it 👍

alexandre-daubois · 2025-06-16T19:33:20Z

@withinboredom yes! As Kevin said, he created an extension that will be revealed tomorrow and that will definitely help you getting started. In the meantime, you can find a some docs here: https://github.com/alexandre-daubois/frankenphp/blob/ext-generator/docs/extensions-generator.md. I need to get back to it to ensure it's up to date with the latest changes, but it covers all supported features if you want to have a look while waiting for tomorrow 🙂

withinboredom · 2025-06-16T20:19:25Z

My biggest feedback:

//export_php:method User::updateInfo(?string $name, ?int $age, ?bool $active): void

reads that it is going to create a static method, not an instanced one. :: usually means static access not instanced, where -> means instanced. As it is currently written, you prevent any way to declare a static method.

edit: I guess there could be a //export_php:static_method or something ... so I guess my comment may not make sense.

withinboredom · 2025-06-16T23:01:35Z

ok, this is pretty neat. In just an hour or two, after reading the docs and fiddling with it, I was able to create an extension from this PR. I feel like this feature still has a looong way to go; but it is already quite powerful. My silly extension just takes a URL and spams it with 50 threads, in parallel, and returns the body by simply calling ->nextBody() (even waits until there is a body if it hasn’t been read yet). I suspect you could do so much with this (e.g. never have to write cURL requests again).

Go really shines with parallel tasks, which is a true pain with PHP (and C, for that matter). I highly suspect that it is only a matter of time before someone writes a Revolt compatible event loop, in Go.

henderkes · 2025-06-17T00:17:50Z

DDOSing in PHP just became so much easier! 💯

Haha, I'm jesting. Another great idea would be to integrate Ristretto because a lot of applications will only need an in-memory cache rather than a shared one. It's on my to-do list after the packaging (unless someone else picks it up first, I see this as revolutionary compared to using APCu with its very slow serialisation and fragmentation issues and... well, necessary system calls). Although it would be very important to benchmark given the very many interop calls and potentially fall back to CacheLib.

Go really shines with parallel tasks, which is a true pain with PHP (and C, for that matter). I highly suspect that it is only a matter of time before someone writes a Revolt compatible event loop, in Go.

Boost cobalt + coroutines come to the rescue. The dependency tree isn't too crazy, it doesn't require the entirety of boost, but of course it comes with the usual C++ style of things. I'd only write the very simplest of extensions in C when using C++ is essentially just as easy on the C API side but gives you all the required tools to write modern, safe code.

withinboredom · 2025-06-17T06:42:59Z

A cache also makes a lot of sense for this.

Although it would be very important to benchmark given the very many interop calls

cgo overhead isn’t too bad, it now only even shows up on flame graphs because we’ve trimmed the fat off of frankenphp into a lean and mean request-processing machine. When frankenphp was slower (i.e. 10–20k rps), you couldn’t even see it on the graph. Most likely, since this can only handle basic value types, you will spend most of the time serializing to string. (though you could also just store an index in the cache that you then retrieve from an array on the PHP side).

withinboredom · 2025-06-17T06:51:04Z

After this PR is merged, here are some things I would like to do to enhance this feature:

add support for property hooks
and static methods
I’m intimately familiar with PHP’s hash map (aka arrays), so I would be happy to figure out how to send them to/from go

types.go

* feat(extensions): add the PHP extension generator * unexport many types * unexport more symbols * cleanup some tests * unexport more symbols * fix * revert types files * revert * add better validation and fix templates * remove GoStringCopy * small fixes --------- Co-authored-by: Kévin Dunglas <kevin@dunglas.fr>

* feat: add helpers to create PHP extensions (#1644) * feat: add helpers to create PHP extensions * cs * feat: GoString * test * add test for RegisterExtension * cs * optimize includes * fix * feat(extensions): add the PHP extension generator (#1649) * feat(extensions): add the PHP extension generator * unexport many types * unexport more symbols * cleanup some tests * unexport more symbols * fix * revert types files * revert * add better validation and fix templates * remove GoStringCopy * small fixes --------- Co-authored-by: Kévin Dunglas <kevin@dunglas.fr> * try to fix tests * fix CS * try some workarounds * try some workarounds * ingore TestRegisterExtension * exclude cgo tests in Docker images * fix * workaround... * race detector * simplify tests and code * make linter happy * feat(gofile): use templates to generate the Go file (#1666) --------- Co-authored-by: Alexandre Daubois <2144837+alexandre-daubois@users.noreply.github.com>

nickchomey · 2025-07-03T22:17:14Z

Another great idea would be to integrate Ristretto because a lot of applications will only need an in-memory cache rather than a shared one. It's on my to-do list after the packaging (unless someone else picks it up first, I see this as revolutionary compared to using APCu with its very slow serialisation and fragmentation issues and... well, necessary system calls). Although it would be very important to benchmark given the very many interop calls and potentially fall back to CacheLib.

This is something that I was also thinking about implementing, as APCu is kind of junk.

I just came across this excellent article that makes a good case for using Otter rather than Ristretto. I'd also like to try NATS KV, which would be similar to the etcd example that kevin made

henderkes · 2025-07-04T01:34:45Z

I gave it a little bit more thought (though I haven't started it yet) and came to the conclusion that writing it in C++ using CacheLib and the existing igbinary extension would likely be the more performant choice. It offers several advantages, such as saving/reloading of cache, no runtime call cost to C, better unwrapping of php types and interop with igbinary, because I wasn't really looking to implement a high performance serialiser on top.

feat(extensions): add the PHP extension generator

f6cf817

alexandre-daubois force-pushed the extension-generator branch 2 times, most recently from f6cf817 to f99078a Compare June 12, 2025 14:08

unexport many types

3e5d07c

dunglas force-pushed the extension-generator branch from f99078a to 3e5d07c Compare June 12, 2025 14:20

alexandre-daubois force-pushed the extension-generator branch from 3e5d07c to 516453f Compare June 12, 2025 14:26

unexport more symbols

f089854

dunglas force-pushed the extension-generator branch from 516453f to f089854 Compare June 12, 2025 14:30

dunglas and others added 6 commits June 12, 2025 16:34

cleanup some tests

ad7064f

unexport more symbols

e71cf21

fix

7d4cf44

revert types files

54468fa

revert

6c7debd

add better validation and fix templates

80b8c61

withinboredom reviewed Jun 13, 2025

View reviewed changes

dunglas requested changes Jun 17, 2025

View reviewed changes

types.go Show resolved Hide resolved

dunglas added 2 commits June 17, 2025 10:55

remove GoStringCopy

bc1f0c7

small fixes

2f98221

dunglas approved these changes Jun 17, 2025

View reviewed changes

dunglas merged commit fcb34c3 into php:ext Jun 17, 2025

dunglas mentioned this pull request Jun 17, 2025

feat: FrankenPHP extensions #1651

Merged

feat(extensions): add the PHP extension generator #1649

feat(extensions): add the PHP extension generator #1649

Uh oh!

Conversation

alexandre-daubois commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PHP Extension Generator

Writing the extension in Go

Type Juggling

What does this generate?

Opaque Classes

Validation

Compiling The Extension

Uh oh!

alexandre-daubois commented Jun 13, 2025

Uh oh!

withinboredom left a comment

Choose a reason for hiding this comment

Uh oh!

withinboredom Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

alexandre-daubois Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

withinboredom Jun 14, 2025

Choose a reason for hiding this comment

Uh oh!

henderkes commented Jun 13, 2025

Uh oh!

withinboredom commented Jun 14, 2025

Uh oh!

henderkes commented Jun 15, 2025

Uh oh!

alexandre-daubois commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

withinboredom commented Jun 16, 2025

Uh oh!

dunglas commented Jun 16, 2025

Uh oh!

AlliBalliBaba commented Jun 16, 2025

Uh oh!

alexandre-daubois commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

withinboredom commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

withinboredom commented Jun 16, 2025

Uh oh!

henderkes commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

withinboredom commented Jun 17, 2025

Uh oh!

withinboredom commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

nickchomey commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

henderkes commented Jul 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

alexandre-daubois commented Jun 12, 2025 •

edited

Loading

alexandre-daubois commented Jun 15, 2025 •

edited

Loading

alexandre-daubois commented Jun 16, 2025 •

edited

Loading

withinboredom commented Jun 16, 2025 •

edited

Loading

henderkes commented Jun 17, 2025 •

edited

Loading

withinboredom commented Jun 17, 2025 •

edited

Loading

nickchomey commented Jul 3, 2025 •

edited

Loading