petsc, py-petsc4py: add v3.18.4, v3.18.5 by balay · Pull Request #36406 · spack/spack

balay · 2023-03-24T14:52:18Z

No description provided.

var/spack/repos/builtin/packages/py-petsc4py/package.py

adamjstewart · 2023-03-24T15:09:32Z

var/spack/repos/builtin/packages/py-petsc4py/package.py


-    depends_on("py-cython", type="build", when="@main")
+    depends_on("py-cython@0.29.32:", when="^python@3.11: @3.19:", type="build")
+    depends_on("py-cython@0.24:", when="@3.19:", type="build")


As is, this says that only main depends on cython.

Yes.

So I can add this change now - or later with the 3.19 release. [as it won't affect current 3.18 releases..]

What I'm saying is that the 3.18 release already contains this constraint in setup.py, but we need to add it to the Spack package too.

3.18 tarballs are prebuilt with cython - so there is no dependency on cython here [when spack builds petsc4py].

As noted - it was only required for @main

With petsc-3.19 - we are not including the prebuilt cython sources in the tarball - hence need this dependency for 3.19 and newer

So I would think that part of setup.py will be skipped in a spack build with petsc4py-3.18 tarballs

cc: @dalcinl to confirm.

Ref: (v3.18) setup.py:

# python-3.11+ requires cython 0.29.32+ if pyver >= (3, 11): CYTHON = '0.29.32' else: CYTHON = '0.24'

We should probably rebuild cython sources even in older versions. This will be done automatically by adding a dependency on cython thanks to #35995. This maximizes the chances that the package will build for all versions of Python.

Why don't you just use the latest Cython release 0.29.33 ?

This is a lower bound, so if a newer version is available it will use that. We want to keep the range of supported versions as wide as possible in case a different package in the DAG has different constraints.

@dalcinl thanks!

the suggested change is pushed

But I don't see cython source generated - even though CYTHON_FORCE_REGEN=1 is set

spack-build-out.txt
spack-build-env.txt

With cython - I would see:

running build_src using Cython 0.29.32 cythonizing 'petsc4py/PETSc.pyx' -> 'petsc4py/PETSc.c' running build_py creating build

Here I'm seeing:

running build running build_src running build_py creating build

Hmm, @haampie any ideas?

tldahlgren

Confirmed the four version sha256.

dalcinl · 2023-03-25T11:16:25Z

Sorry, guys... Are you aware that regenerating Cython sources in old petsc4py releases will require "manual" step? See how it is done for mpi4py in conda-forge. Running that exact same line on petsc4py sources is the way to go.

PS: Sorry if you are aware of this, but looking at Spack's py-petsc4py/package.py, it is not obvious things are done the way it should be done.

balay · 2023-03-25T15:49:57Z

> {{ PYTHON }} conf/cythonize.py

I'll defer the spack impl code for this to @adamjstewart

Perhaps this cython issue should be decoupled from this PR [and #36401 ] - this build change be applied to mpi4py,petsc4py,slepc4y in a separate PR

adamjstewart · 2023-03-25T16:44:19Z

I'm fine with applying this in a separate PR. The code would look like:

python(join_path("conf", "cythonize.py"))

balay · 2023-03-25T17:01:53Z

python(join_path("conf", "cythonize.py"))

Does it go into def build()?

What is the current default build command that it invokes? presumably I would have to replicate it in build()?

balay · 2023-03-25T17:27:50Z

Ah - ok - I see it should be install(). Perhaps?

    def install(self, spec, prefix):
        with working_dir(self.build_directory):
            python(join_path("conf", "cythonize.py"))
            args = std_pip_args + ["--prefix=" + prefix, "."]
            pip(*args)

I now get:

==> py-petsc4py: Executing phase: 'install'
==> [2023-03-25-12:22:01.859652] '/opt/intel/oneapi/intelpython/python3.9/bin/python3.9' 'conf/cythonize.py'
==> [2023-03-25-12:22:09.924187] '/opt/intel/oneapi/intelpython/python3.9/bin/python3.9' '-m' 'pip' '-vvv' '--no-input' '--no-cache-dir' '--disable-pip-version-check' 'install' '--no-deps' '--ignore-installed' '--no-build-isolation' '--no-warn-script-location' '--no-index' '--prefix=/home/balay/spack/opt/spack/linux-fedora37-skylake/gcc-12.2.1/py-petsc4py-3.18.5-uy6i3q6kwn6bzmjnl2whreswqsgznxgk' '.'

Presumably its working?

balay · 2023-03-25T17:34:13Z

pushed this change

adamjstewart · 2023-03-25T21:18:41Z

var/spack/repos/builtin/packages/py-petsc4py/package.py

        else:
            return self.stage.source_path
+
+    def install(self, spec, prefix):


I would probably do this as:

@run_before("install") def cythonize(self): python(join_path("conf", "cythonize.py"))

Thanks! updated now.

@main

* py-petsc4py: update ldshared-dev.patch [to work with current @main] * petsc4py: always force rebuild cython sources

…y release tarball (that depend on cython < 0.29.36) This avoids conflict with petsc4py versions that have a dependency on cython@3.0.0 - revert #36406, #38996 - update cython dependency for mpi4py@master

petsc, py-petsc4py: add v3.18.4, v3.18.5

ed3629c

balay requested a review from adamjstewart March 24, 2023 14:52

spackbot-app bot added new-version python update-package labels Mar 24, 2023

spackbot-app bot requested review from BarrySmith, jedbrown and pradyunsg March 24, 2023 14:52

adamjstewart reviewed Mar 24, 2023

View reviewed changes

var/spack/repos/builtin/packages/py-petsc4py/package.py Show resolved Hide resolved

spackbot-app bot added the dependencies label Mar 24, 2023

adamjstewart reviewed Mar 24, 2023

View reviewed changes

balay force-pushed the balay/petsc-3.18.5 branch from 6ded227 to 374df89 Compare March 24, 2023 15:09

balay mentioned this pull request Mar 24, 2023

New patch release SLEPc 3.18.3 #36401

Merged

tldahlgren assigned adamjstewart Mar 24, 2023

tldahlgren reviewed Mar 24, 2023

View reviewed changes

tldahlgren previously approved these changes Mar 24, 2023

View reviewed changes

tldahlgren self-assigned this Mar 24, 2023

py-petsc4py: update ldshared-dev.patch [to work with current @main]

f7b3f08

balay dismissed tldahlgren’s stale review via 2bb7f7b March 24, 2023 19:06

balay force-pushed the balay/petsc-3.18.5 branch from 770bc6b to 2bb7f7b Compare March 24, 2023 19:06

adamjstewart previously approved these changes Mar 25, 2023

View reviewed changes

balay dismissed adamjstewart’s stale review via 746aca5 March 25, 2023 17:34

balay force-pushed the balay/petsc-3.18.5 branch from 2bb7f7b to 746aca5 Compare March 25, 2023 17:34

adamjstewart reviewed Mar 25, 2023

View reviewed changes

petsc4py: always force rebuild cython sources

9c9bddd

balay force-pushed the balay/petsc-3.18.5 branch from 746aca5 to 9c9bddd Compare March 26, 2023 06:41

adamjstewart approved these changes Mar 26, 2023

View reviewed changes

balay mentioned this pull request Mar 27, 2023

py-mpi4py: always force rebuild cython sources #36460

Merged

alalazo merged commit 56d98c3 into develop Mar 27, 2023

alalazo deleted the balay/petsc-3.18.5 branch March 27, 2023 07:44

eugeneswalker pushed a commit to eugeneswalker/spack that referenced this pull request May 15, 2023

petsc, py-petsc4py: add v3.18.4, v3.18.5 (spack#36406)

638ab6f

* py-petsc4py: update ldshared-dev.patch [to work with current @main] * petsc4py: always force rebuild cython sources

balay mentioned this pull request Aug 28, 2023

Next release? 4.0.0? mpi4py/mpi4py#386

Closed

Conversation

balay commented Mar 24, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

balay Mar 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

balay Mar 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tldahlgren left a comment

Choose a reason for hiding this comment

Uh oh!

dalcinl commented Mar 25, 2023

Uh oh!

balay commented Mar 25, 2023

Uh oh!

adamjstewart commented Mar 25, 2023

Uh oh!

balay commented Mar 25, 2023

Uh oh!

balay commented Mar 25, 2023

Uh oh!

balay commented Mar 25, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

balay Mar 24, 2023 •

edited

Loading

balay Mar 24, 2023 •

edited

Loading