Add support for Python 3.12 by cbrnr · Pull Request #240 · holgern/pyedflib

cbrnr · 2023-10-03T06:19:47Z

Maybe this already works, let's see.

cbrnr · 2023-10-03T07:13:13Z

It seems like setup-python doesn't have the final 3.12 release yet, so let's wait until this happens.

cbrnr · 2023-10-04T08:46:38Z

Python 3.12 should now be available. @skjerns could you restart the jobs please (I don't have permissions to do that myself)?

skjerns · 2023-10-04T08:58:22Z

perfect, thanks :)

cbrnr · 2023-10-04T09:06:49Z

Any chance for a new release so that people can use pyedflib with Python 3.12?

skjerns · 2023-10-04T09:15:57Z

sure :) https://github.com/holgern/pyedflib/releases/tag/v0.1.35

skjerns · 2023-10-04T20:38:35Z

Ah damn, it doesn't seemt to build for 3.12

https://github.com/holgern/pyedflib/actions/runs/6403993363/job/17383647330#step:5:4202

      Downloading numpy-1.26.0.tar.gz (15.6 MB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.6/15.6 MB 90.4 MB/s eta 0:00:00
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Installing backend dependencies: started
      Installing backend dependencies: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'error'
      error: subprocess-exited-with-error
  
      × Preparing metadata (pyproject.toml) did not run successfully.
      │ exit code: 1
      ╰─> [58 lines of output]
          + /opt/python/cp312-cp312/bin/python /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/vendored-meson/meson/meson.py setup /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build/meson-python-native-file.ini
          The Meson build system
          Version: 1.2.99
          Source dir: /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc
          Build dir: /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build
          Build type: native build
          Project name: NumPy
          Project version: 1.26.0
          C compiler for the host machine: cc (gcc 10.2.1 "cc (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)")
          C linker for the host machine: cc ld.bfd 2.35-5
          C++ compiler for the host machine: c++ (gcc 10.2.1 "c++ (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)")
          C++ linker for the host machine: c++ ld.bfd 2.35-5
          Cython compiler for the host machine: cython (cython 3.0.2)
          Host machine cpu family: x86
          Host machine cpu: i686
          Program python found: YES (/opt/python/cp312-cp312/bin/python)
          Found pkg-config: /usr/bin/pkg-config (0.27.1)
          Run-time dependency python found: YES 3.12
          Has header "Python.h" with dependency python-3.12: YES
          Compiler for C supports arguments -fno-strict-aliasing: YES
          Test features "SSE SSE2" : Supported
          Test features "SSE3" : Supported
          Test features "SSSE3" : Supported
          Test features "SSE41" : Supported
          Test features "POPCNT" : Supported
          Test features "SSE42" : Supported
          Test features "AVX" : Supported
          Test features "F16C" : Supported
          Test features "FMA3" : Supported
          Test features "AVX2" : Supported
          Test features "AVX512F" : Supported
          Test features "AVX512CD" : Supported
          Test features "AVX512_KNL" : Supported
          Test features "AVX512_KNM" : Supported
          Test features "AVX512_SKX" : Supported
          Test features "AVX512_CLX" : Supported
          Test features "AVX512_CNL" : Supported
          Test features "AVX512_ICL" : Supported
          Test features "AVX512_SPR" : Unsupported due to Arguments "-msse, -msse2, -msse3, -mssse3, -msse4.1, -mpopcnt, -msse4.2, -mavx, -mf16c, -mfma, -mavx2, -mno-mmx, -mavx512f, -mavx512cd, -mavx512vl, -mavx512bw, -mavx512dq, -mavx512vnni, -mavx512ifma, -mavx512vbmi, -mavx512vbmi2, -mavx512bitalg, -mavx512vpopcntdq, -mavx512fp16" are not supported
          Configuring npy_cpu_dispatch_config.h using configuration
          Message:
          CPU Optimization Options
            baseline:
              Requested : min
              Enabled   : SSE SSE2
            dispatch:
              Requested : max -xop -fma4
              Enabled   : SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
  
          Library m found: YES
          Found CMake: /usr/local/bin/cmake (3.27.6)
          WARNING: CMake Toolchain: Failed to determine CMake compilers state
          Run-time dependency openblas found: NO (tried pkgconfig and cmake)
          Run-time dependency openblas found: NO (tried pkgconfig and cmake)
  
          ../../numpy/meson.build:207:4: ERROR: Problem encountered: No BLAS library detected! Install one, or use the `allow-noblas` build option (note, this may be up to 100x slower for some linear algebra operations).
  
          A full log can be found at /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build/meson-logs/meson-log.txt
          [end of output]
  
      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: metadata-generation-failed
  
    × Encountered error while generating package metadata.
    ╰─> See above for output.
  
    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.
    error: subprocess-exited-with-error
    
    × pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /opt/python/cp312-cp312/bin/python /opt/_internal/cpython-3.12.0/lib/python3.12/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-09389xue/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools oldest-supported-numpy 'cython<3.0'
    cwd: [inherit]
    Installing build dependencies: finished with status 'error'
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

cbrnr · 2023-10-05T04:27:07Z

Why is it trying to build NumPy from source?

skjerns · 2023-10-05T07:52:56Z

I think for building the Cython extension with cimports numpy.
I'm sure there is/was a reason for that, but I don't know whether it's still necessary and could be done elsewise. It was setup like that before I joined the project and I never looked into it.

cbrnr · 2023-10-05T08:39:19Z

For some reason, it does not find a BLAS library:

https://github.com/holgern/pyedflib/actions/runs/6403993363/job/17383647330#step:5:4169

Why do the other build < 3.12 work then? Don't they need one (e.g. openblas)? Maybe explicitly installing openblas could solve the issue?

skjerns · 2023-10-05T10:10:27Z

Good question. Maybe not available on the Python 3.12 runner?

cbrnr · 2023-10-05T12:14:27Z

It should be the same ubuntu-latest image for all Python versions I think. Unless cibuildwheel is using their own container images for each Python version. It's worth a shot trying to install openblas just for Python 3.12 I think.

cbrnr · 2023-10-05T16:10:14Z

The container image for x86 is the same for all Python versions (manylinux2014_i686). Everything works on manylinux2014_x86_64, including Python 3.12. I can't read the detailed log (it says A full log can be found at /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build/meson-logs/meson-log.txt), because I don't know to access that location from the results website.

I see two options that require only little work:

Consider dropping the i686 architecture completely. Almost all major packages only build for x86_64.
Skip Python 3.12 for i686.

Figuring out why the build for 3.12 on i686 is failing will require more digging around. Personally, I'd go with option 1 (people requiring the package for i686 can always build from source).

skjerns · 2023-10-06T14:57:11Z

This seems to be related: numpy/numpy#24703

Even though I agree that x86 can probably be dropped, I'd rather leave it, who knows on which devices this is running. I'll try to explicitly add openblas to python>3.11, else Skipping i686 for >3.11 would be the best solution I'd say.

skjerns · 2023-10-08T19:15:10Z

finally it worked!

new release it out

cbrnr · 2023-10-09T04:46:12Z

Nice! Why though? 😀

skjerns · 2023-10-09T05:50:26Z

Nice! Why though? 😀

What do you mean?

cbrnr · 2023-10-09T06:34:07Z

I mean, why is it working now? Did you change anything?

skjerns · 2023-10-09T06:53:32Z

Ah, sorry, I meant: finally excluding x68 for py3.12 worked. Installing openblas did not solve it unfortunately.

I had to wiggle the settings for the wheels.yaml quite a bit until I found out a setting that had the intended effect.

this worked:

          CIBW_SKIP: "pp* cp312*i686"
          CIBW_PRERELEASE_PYTHONS: False

Add support for Python 3.12

eb4b094

skjerns merged commit 49bfedd into holgern:master Oct 4, 2023

cbrnr deleted the python312 branch October 4, 2023 09:00

Conversation

cbrnr commented Oct 3, 2023

Uh oh!

cbrnr commented Oct 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cbrnr commented Oct 4, 2023

Uh oh!

skjerns commented Oct 4, 2023

Uh oh!

cbrnr commented Oct 4, 2023

Uh oh!

skjerns commented Oct 4, 2023

Uh oh!

skjerns commented Oct 4, 2023

Uh oh!

cbrnr commented Oct 5, 2023

Uh oh!

skjerns commented Oct 5, 2023

Uh oh!

cbrnr commented Oct 5, 2023

Uh oh!

skjerns commented Oct 5, 2023

Uh oh!

cbrnr commented Oct 5, 2023

Uh oh!

cbrnr commented Oct 5, 2023

Uh oh!

skjerns commented Oct 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

skjerns commented Oct 8, 2023

Uh oh!

cbrnr commented Oct 9, 2023

Uh oh!

skjerns commented Oct 9, 2023

Uh oh!

cbrnr commented Oct 9, 2023

Uh oh!

skjerns commented Oct 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cbrnr commented Oct 3, 2023 •

edited

Loading

skjerns commented Oct 6, 2023 •

edited

Loading

skjerns commented Oct 9, 2023 •

edited

Loading