Skip to content

Preliminary Python 3.14t support#12223

Merged
crusaderky merged 1 commit intodask:mainfrom
crusaderky:314t
Feb 4, 2026
Merged

Preliminary Python 3.14t support#12223
crusaderky merged 1 commit intodask:mainfrom
crusaderky:314t

Conversation

@crusaderky
Copy link
Copy Markdown
Collaborator

@crusaderky crusaderky commented Jan 6, 2026

  • Run test suite on 3.14t, without pandas for now. While pandas 3 in theory supports free-threading, there are many race conditions left in 3.0.0rc1. [EDIT] support for pandas >=3.0.1 added in Pandas in 3.14t CI #12284
  • msgpack, which is a key dependency of distributed, currently needs a very ugly hack. This is being tracked upstream (Builds on no-GIL interpreters, but not fully supported yet msgpack/msgpack-python#613). As I was forced to install it in pure-python mode, I would advise against running benchmarks until the C extension becomes available.
  • Many other optional dependencies are still missing and will be added as they become available.
  • Added pypi classifier Programming Language :: Python :: Free Threading :: 1 - Unstable. Technically it's known broken for dask.dataframe and 2 - Beta for everything else. I will bump the flag to 2 - Beta once pandas fixes the race conditions upstream and we can reintroduce it to the CI environment. Also probably a good idea to wait for a hack-free msgpack before we indicate a good user experience.

This PR incorporates and is blocked by #12224. This PR only adds the last commit 208891b.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jan 6, 2026

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

     21 files  +     2       21 suites  +2   5h 13m 39s ⏱️ + 9m 28s
 18 282 tests ±     0   17 010 ✅ ±     0   1 272 💤 ±    0  0 ❌ ±0 
294 733 runs  +13 460  252 433 ✅ +11 892  42 300 💤 +1 568  0 ❌ ±0 

Results for commit a642b8d. ± Comparison against base commit 08f5daa.

♻️ This comment has been updated with latest results.

@crusaderky
Copy link
Copy Markdown
Collaborator Author

CI failures are unrelated.
Please review and merge #12224 first.

@crusaderky crusaderky marked this pull request as ready for review January 7, 2026 10:43
@ngoldbaum
Copy link
Copy Markdown

ngoldbaum commented Jan 13, 2026

I see this failure that looks pertinent when I try to run the dask unit tests locally on my ARM Mac on this PR branch with free-threaded 3.14:

FAILED dask/array/tests/test_array_core.py::test_store_locks - assert 49 == 50
 +  where 49 = <dask.array.tests.test_array_core.CounterLock object at 0x24b5bd37950>.acquire_count
 +  and   50 = <dask.array.tests.test_array_core.CounterLock object at 0x24b5bd37950>.release_count

I also see a few other test failures that look unrelated:

FAILED dask/array/tests/test_array_core.py::test_arithmetic - DeprecationWarning: numpy.fix is deprecated. Use numpy.trunc instead, which is faster and follows the Array API standard.
FAILED dask/dataframe/dask_expr/tests/test_collection.py::test_repartition_partition_size - AssertionError: assert 5 == 4
 +  where 5 = Dask DataFrame Structure:\n                   x      y\nnpartitions=5              \n0              int64  int64\n20               ...    ...\n...              ...    ...\n80               ...    ...\n99               ...    ...\nDask Name: repartition, 2 expressions\nExpr=Repartition(frame=df, partition_size='1kb').npartitions
FAILED dask/dataframe/io/tests/test_parquet.py::test_arrow_partitioning - pandas.errors.Pandas4Warning: Constructing a Categorical with a dtype and values containing non-null entries not in that dtype's categories is deprecated and will raise in a future version.
FAILED dask/dataframe/io/tests/test_parquet.py::test_roundtrip_partitioned_pyarrow_dataset[pyarrow] - KeyError: 2

I got impatient and killed the tests in dask/dataframe/tests/test_categorical.py so I didn't actually run the full test suite.

@crusaderky
Copy link
Copy Markdown
Collaborator Author

FAILED dask/array/tests/test_array_core.py::test_store_locks - assert 49 == 50

  • where 49 = <dask.array.tests.test_array_core.CounterLock object at 0x24b5bd37950>.acquire_count
  • and 50 = <dask.array.tests.test_array_core.CounterLock object at 0x24b5bd37950>.release_count

Are you on numpy git tip? Do you get the same failures with 2.4.1? What about the nightly wheels?

FAILED dask/array/tests/test_array_core.py::test_arithmetic - DeprecationWarning: numpy.fix is deprecated. Use numpy.trunc instead, which is faster and follows the Array API standard.

Thanks; I think the reason CI didn't spot this yet is that numpy nightly wheels are lagging a bit behind git tip?
#12231

FAILED dask/dataframe/dask_expr/tests/test_collection.py::test_repartition_partition_size - AssertionError: assert 5 == 4
where 5 = Dask DataFrame Structure:\n x y\nnpartitions=5 \n0 int64 int64\n20 ... ...\n... ... ...\n80 ... ...\n99 ... ...\nDask Name: repartition, 2 expressions\nExpr=Repartition(frame=df, partition_size='1kb').npartitions
FAILED dask/dataframe/io/tests/test_parquet.py::test_arrow_partitioning - pandas.errors.Pandas4Warning: Constructing a Categorical with a dtype and values containing non-null entries not in that dtype's categories is deprecated and will raise in a future version.
FAILED dask/dataframe/io/tests/test_parquet.py::test_roundtrip_partitioned_pyarrow_dataset[pyarrow] - KeyError: 2

Any idea why these don't show up on 3.14 with pandas nightly? Similar lag issue as above?
#12225

@ngoldbaum
Copy link
Copy Markdown

ngoldbaum commented Jan 13, 2026

Are you on numpy git tip? Do you get the same failures with 2.4.1? What about the nightly wheels?

Yup, NumPy is built from source. The failure seems to be flaky, it only happened the first time I ran the tests not the second time after I installed optional dependencies. Now that I'm trying just that one test repeatedly, it seems to fail about half of the times I try to run it. I can trigger it with the 2.4.1 NumPy wheel from PyPI.

FWIW, I see no crashes in Pandas. Maybe try uncommenting Pandas to see if any of the CI runners hit the crash?

Thanks; I think the reason CI didn't spot this yet is that numpy nightly wheels are lagging a bit behind git tip?
#12231

Yeah, the deprecation just got merged today so no surprise there.

Any idea why these don't show up on 3.14 with pandas nightly? Similar lag issue as above?

No idea, sorry.

@crusaderky
Copy link
Copy Markdown
Collaborator Author

@jacobtomlinson @jsignell does either of you have bandwidth to offer review?
(note that this is blocked by and incorporates #12224)

@crusaderky
Copy link
Copy Markdown
Collaborator Author

With pandas 3.0 final I'm getting these failures

FAILED dask/dataframe/dask_expr/tests/test_collection.py::test_repartition_partition_size - AssertionError: assert 5 == 4
FAILED dask/tests/test_tokenize.py::test_tokenize_range_index - assert 16 >= 32

I'd rather investigate them in a follow-up to avoid scope creep.

Copy link
Copy Markdown
Member

@jsignell jsignell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, I approved #12224 (review) as well, but just have the question about performance.

@crusaderky crusaderky merged commit 1be4042 into dask:main Feb 4, 2026
30 checks passed
@crusaderky crusaderky deleted the 314t branch February 4, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants