Skip to content

Use meson build backend for scipy#4276

Merged
ryanking13 merged 12 commits intopyodide:mainfrom
ryanking13:scipy-meson2
Nov 17, 2023
Merged

Use meson build backend for scipy#4276
ryanking13 merged 12 commits intopyodide:mainfrom
ryanking13:scipy-meson2

Conversation

@ryanking13
Copy link
Copy Markdown
Member

Description

Trying to use meson build backend when building scipy.

Close #3649

@hoodmane
Copy link
Copy Markdown
Member

hoodmane commented Nov 1, 2023

Thanks so much for your work on meson @ryanking13!

@ryanking13 ryanking13 marked this pull request as ready for review November 2, 2023 11:21
@ryanking13
Copy link
Copy Markdown
Member Author

Okay, I think now CI passes.

@ryanking13
Copy link
Copy Markdown
Member Author

ryanking13 commented Nov 6, 2023

cc: @lesteve

I hope not, but this may break something. Please let me know if something goes wrong after merging this PR.

@lesteve
Copy link
Copy Markdown
Contributor

lesteve commented Nov 7, 2023

Sounds good, currently Scipy is a bit broken on main (or at least a bit more broken than previously) due to #4249 but I'll let you know in case I notice additional issues.

@lesteve
Copy link
Copy Markdown
Contributor

lesteve commented Nov 7, 2023

Side-comment, it would probably be worth doing a PR on conda-forge/pkg-config-feedstock to bump the build number and have a package available with the fix from conda-forge/pkg-config-feedstock#31

@lesteve
Copy link
Copy Markdown
Contributor

lesteve commented Nov 14, 2023

Please let me know if something goes wrong after merging this PR.

FYI I ran the scipy tests locally on this PR and they pass, i.e. this PR doesn't break anything else than what is already broken in main. I had a look at the changes and they seem fine.

All in all, I think this can be merged!

@ryanking13
Copy link
Copy Markdown
Member Author

Oh, thanks so much for the check @lesteve!

Copy link
Copy Markdown
Member

@hoodmane hoodmane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ryanking13!

for lib in ["lapack", "blas"]:
if path.endswith(f"cython_{lib}.c"):
header_path = Path(path).with_name(f"_{lib}_subroutines.h")
header_name = f"_{lib}_subroutines.h"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put this in a separate find_header function?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe this is fine as is...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it to a separate function. Thanks for the suggestion!

Comment on lines +682 to +683
if tmp is not None:
line = tmp
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain this change? In particular we used to quit out here if replay_f2c returns None but now we don't? Why? When does this happen? What is the subprocess call below going to do in this case?

Also, we should switch handle_command to return an int since in compile_main it says sys.exit(handle_command(args, build_args)) we can just return 0 or return returncode in here.

Copy link
Copy Markdown
Member Author

@ryanking13 ryanking13 Nov 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in another comment, meson runs gfortran multiple times during the compiler check phase, so sometimes replay_f2c doesn't do anything. So I changed it to run gfortran normally in that case. That is why I removed the check for -dumpversion, as it will be handled as well.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we should switch handle_command to return an int since in compile_main it says sys.exit(handle_command(args, build_args)) we can just return 0 or return returncode in here.

Right, thanks for the pointer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the following work:

tmp = replay_f2c(line)
if tmp is None:
   # No source file, it's a query for information about the compiler. Pretend we're
   # gfortran by letting gfortran handle it
   return subprocess.run(line).returncode
line = tmp

Or is it important that we are calling handle_command_generate_args on it? I think it would help to have some comments here since this is a really confusing execution path.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your code will work. Thanks for the suggestion.

since this is a really confusing execution path.

I agree. Actually pywasmcross is becoming more complicated as we add more symlinks... I'll try do to some refactorings when I have bandwidth.

extra_vars = set(
["PYODIDE", "PKG_CONFIG_PATH", "CMAKE_TOOLCHAIN_FILE", "PYO3_CONFIG_FILE"]
)
extra_vars = set(["PYODIDE", "CMAKE_TOOLCHAIN_FILE", "PYO3_CONFIG_FILE"])
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason PKG_CONFIG_LIBDIR doesn't belong here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These variables are for checking extra variables that does not come from Makefile.envs but are added from the code . So PKG_CONFIG_PATH or PKG_CONFIG_LIBDIR does not need to belong there.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment like:

# extra variables that does not come from Makefile.envs but are added by build_env.py

@ryanking13 ryanking13 merged commit 071fe35 into pyodide:main Nov 17, 2023
@ryanking13 ryanking13 deleted the scipy-meson2 branch November 17, 2023 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants