Towards a python.util.concurrent?

Ideas
1

Hello, first post here :waving_hand:

I’ve just released a first version of GitHub - dpdani/cereggii: Thread synchronization utilities for Python to PyPI, and I’m looking for feedback! :white_flower:

Looking at @colesbury’s nogil, and the work currently undergoing to port it to 3.13, I’ve been working on a sort of java.util.concurrent-like package for free-threaded Python (I’m referring to java here just because I know it better), and I’m currently working on making some atomic lock-free data structures, as part of my MSc thesis.

Let me explain why I think atomic lock-free data structures may be useful for Python.

Consider a program (or some phase of execution of a program) in which all threads are concurrently trying to modify a shared object.

For instance, consider an aggregation routine, where

  1. threads that are reading data, i.e. doing I/O;
  2. read the current value of the aggregation, i.e. read from a dict;
  3. compute the updated value; and
  4. try to update the aggregate, i.e. write to a dict.

In this program, free-threaded CPython provides an improvement over non-free-threaded CPython only in so far as point (3) above is concerned.
Notice the behavior of blocking I/O in (1) does not change, compared to CPython with the GIL.

Points (2), and (4) instead provide no improvement under free-threaded CPython because the accesses around the dict itself require holding a lock, specific to the dict instance.

Compared to free-threaded CPython, the data types in cereggii provide some notable performance improvements. Consider for instance these examples: AtomicInt by dpdani · Pull Request #7 · dpdani/cereggii · GitHub

There are also other examples in the README, with more detailed explanations.

A question on packaging

When a user downloads my package cereggii, I’d like to tell that a free-threaded CPython is required to run it.
I currently do this by raising a warning at runtime if not getattr(sys.flags, "nogil", False) (and implicitly when the import raises a bunch of other warnings).
I’m currently basing my work on Sam’s original nogil implementation, and I’d like to eventually rebase on 3.13 when the first beta comes out.

My question is, when I do so, how can I express in my pyproject.toml that I specifically require a free-threaded interpreter?
Has this issue been discussed already?

P.S. I’m only distributing source right now (PyPI won’t allow the nogil39 tag), and I’m pretty sure this will not compile on Arm processors.

5 Likes
2

There are ongoing discussions about how we want to identify the different 3.13 builds from Python code. I think there’s a good chance that some identifier will end up in sys.thread_info. You can also use sysconfig.get_config_var("Py_GIL_DISABLED").

2 Likes
3

That’s interesting.

Although, I guess my question was more about the front-end: how would pip/poetry/etc know that this cereggii package will not be able to be imported if the Python build is not free-threaded?

It would be nice for pip, or any other tool, to fail to install the package instead, so that the user installing it knows about the dependency on the interpreter build type.

Has there been a discussion on how pip may identify packages that require the free-threaded changes?

4

My understanding is that the ABI (which is a component of the wheel filename) will be different for free-threaded builds.

5

Ah right, makes sense.
But would a package for Python 3.13 be allowed to support the free-threaded build and not the standard build as well?

AFAIK, if pip fails to find a matching wheel, it will try to compile from the source distribution. But this package’s sdist is not intended to support non-free-threaded builds.

As it stands, when PyPI will allow, I could try to upload two wheels, one for 3.13t and another for 3.13 which would just raise an error.
Would this be the expected behavior for my use case?

6

Edit: My understanding was incorrect; see below.

Expand to see my original post

My understanding is that free-threaded-only wheels won’t be supported packaging metadata.
AFAIU:

  • The 3.13 ABI will be the same between threaded and non-threaded builds.
  • Free-threaded builds will be able to turn the GIL on again, so sysconfig or packaging tools can’t reliably tell you whether the GIL isn’t used.
  • As soon as free-threaded builds stop being experimental, they’ll become the default build.

I don’t think there’s any plan to adapt PyPI and packaging tools specifically for the experimental builds.
As I understand it, the ABI tags might be:

  • cp313: compatible with 3.13 only (both default and free-threaded)
  • abi3: compatible with 3.2+, non-free-threaded only
  • abi4 (currently only a preliminary idea): compatible with 3.9+ or so, both default and free-threaded
7

There seems to be some confusion around this, is there something official yet?

Not having two different tags would make it impossible for a package with a C extension to provide two different builds. Though I guess it would make sense in light of this:

But if the ABIs themselves don’t change, then it’s only a problem in so far as the library behaves around the presence of the GIL and it wouldn’t necessarily break.
Then, for the use case of my library it would make little sense to use it if the GIL is enabled, but that can only be determined at runtime, so keeping a warning seems to be the best option I’ll have, also after the release of 3.13.

So, if there are multiple interpreters, and possibly only some of them have the GIL disabled (are free-threaded), the warning check should be run in several places, not just at the module initialization.
For instance in cereggii, the check should be performed when a thread calls AtomicInt.__iadd__, and other similar spots.

In order to give the notice though, there should be a per-interpreter API to be called, is there one? I don’t think the sys.thread_info that @colesbury mentioned provides this information.

8

What @pf_moore wrote is correct. The wheel tag is different for the 3.13 free-threaded builds. This is described in the PEP and implemented in the main branch.

Some of what @encukou wrote is not accurate: the default and free-threaded builds are not ABI compatible. The tags are:

  • cp313: compatible with 3.13 default build (with GIL) only
  • cp313t: compatible with 3.13 free-threaded build (no GIL) only
  • abi3: compatible with 3.2+, non-free-threaded only

It is straightforward to only make your extension work in free-threaded builds – only upload a wheel built with the free-threaded CPython build – but I would discourage you from doing so:

  1. Some of the classes you provide are useful even with the GIL. There’s currently no atomic int in Python and people go through all sorts of weird contortions to atomically count things (which often leads to broken code even with the GIL)
  2. You are providing an extension that other libraries will build on top of. Those libraries may want to support both the default and free-threaded builds of CPython. Artificially limiting your extension makes things harder for those building on top of your extension unnecessarily.

Of course, ultimately what you want to support and how you communicate it is up to you.

2 Likes
9

Also, if there is no wheel for a GIL build, but there is a sdist, pip will currently try to build a wheel (unless you set --only-binary), so the UX is poor. We’re looking at ways to improve it, but that’s a consideration for now at least.

10

Maybe this thread is the wrong place to discuss this, but IIRC at the October core dev sprint in Brno, didn’t we discuss changes to the ABI to make it compatible with both free-threaded and default builds? Or was that only for the stable ABI?

11

That was only for the stable ABI – what Petr is referring to as abi4. And as Petr wrote, it’s still a preliminary idea: no code or PEP yet.

1 Like
12

You’re right, I see your point. Though it requires me to support both ABIs which I don’t currently do.

I think I will need to review the differences between the two ABIs, is there some documentation already?
I’m especially interested in the functions that were added into pyatomic_gcc.h (and its siblings), which are not part of the stable ABI. (Is there a particular reason they aren’t?)
If my understanding is correct, they won’t be available in the default build.

Speaking of ABIs, I would also need a _Py_atomic_compare_exchange_int128 for some optimizations, which is not in pyatomic_*.h and is only supported on some platforms. I can draft a PR to add it if you think it could be accepted; perhaps the availability could be checked in ./configure?
Or else, I can keep using the gcc builtin directly.

In general, I think I’ll hold on trying to support both ABIs in my library until the first 3.13 beta is out.

Would be keen on testing the improvements!

13

It would most likely consist of making --only-binary :all: the default, so you can try it out right now :slightly_smiling_face:

1 Like
14

No, not yet, but the differences in ABI are not too relevant for extension authors beyond the fact that extensions compiled with the free-threaded build can’t be used with the default build and vice-versa.

These functions are not part of the public API. The leading underscore indicates they are private. There is a discussion on making the APIs public, but it did not reach a resolution.

I’d encourage you to use C11 (or C++11) atomics directly. You can even support MSVC if you compile with /experimental:c11atomics and a recent MSVC version. This will allow you to support older CPython versions that don’t include the pyatomic.h headers. (And to be clear – the functions in pyatomic.h header are not public – they are not intended to be used by extensions currently).

That make sense. I don’t think the --disable-gil 3.13 builds are in a state yet to try out either – they still have the GIL, for example.

15

Right, missed that.

Well, I think it makes a lot of sense for my library to do a sort of copy-paste of those functions then. I was interested in using those directly because I don’t want to restrict the compiler choice of a user that downloads and builds the sdist of cereggii, and the supported compilers of CPython seemed a large enough selection already.

In the coming future I’ll switch to having them inside my package. I’ll keep an eye on the discussion you linked and if Python decides to expose those functions then I may switch to using them again. Unless back-porting them to earlier versions of Python becomes interesting for cereggii.

Btw, thanks for pointing out the library could be used also with the GIL enabled, didn’t consider those use cases before. :folded_hands:

16

Maybe it’s the wrong place to ask, but will _Py_TRY_INCREF be public? It’s very crucial to implement correct concurrent reference counting.