Add GraalPy support by timfel · Pull Request #5141 · astral-sh/uv

timfel · 2024-07-17T08:48:47Z

Summary

Currently, uv refuses to install anything on GraalPy. This is currently blocking GraalPy testing with cibuildwheel, since manylinux includes both uv and graalpy (but doesn't test with uv), whereas cibuildwheel defaults to uv. See e.g. https://github.com/pypa/cibuildwheel/actions/runs/9956369360/job/27506182952?pr=1538 where it gives

      + python -m build /project/sample_proj --wheel --outdir=/tmp/cibuildwheel/built_wheel --installer=uv
  * Creating isolated environment: venv+uv...
  * Using external uv from /usr/local/bin/uv
  * Installing packages in isolated environment:
    - setuptools >= 40.8.0
  > /usr/local/bin/uv pip install "setuptools >= 40.8.0"
  < error: Unknown implementation: `graalpy`

Test Plan

I simply based the GraalPy support on PyPy and added some small tests. I'm open to discussing how to test this. GraalPy is available for manylinux images and with setup-python, so we should be able to add tests against it to the CI. I locally confirmed by installing uv into a GraalPy venv and then trying things like uv pip install Pillow and testing those extensions.

crates/uv-virtualenv/src/bare.rs

zanieb · 2024-07-17T17:33:14Z

Thanks for contributing! Welcome to the project :)

Could you add a CI integration test in the style of the PyPy one?

timfel · 2024-07-18T07:59:11Z

Could you add a CI integration test in the style of the PyPy one?

Added. There's a bug with GraalPy 24.0 on Linux around parsing the home property in pyvenv.cfg that we hadn't noticed so far because the venv and virtualenv modules seem to be doing something different. I fixed this in GraalPy master already and the fix is due out in the first week of September. Until then I added a workaround to the CI job for GraalPy on Linux.

timfel · 2024-07-18T07:59:59Z

.github/workflows/ci.yml

+    runs-on: windows-latest
+
+    steps:
+      - uses: timfel/setup-python@fc9bcb4a04f5b1ea7d678c2ca7ea1c479a2468d7


PR to setup-python is open: actions/setup-python#880

timfel · 2024-07-18T08:00:13Z

.github/workflows/ci.yml

+      - name: Graalpy info
+        run: |
+          which graalpy
+          echo "GRAAL_PYTHONHOME=$(graalpy -c 'print(__graalpython__.home)')" >> $GITHUB_ENV


Workaround for 24.0 bug (fixed in 24.1, due out in September)

zanieb · 2024-07-18T14:42:38Z

Looks like it's failing on Windows: https://github.com/astral-sh/uv/actions/runs/9987705348/job/27602873156?pr=5141

zanieb

Looks good to me once it's passing CI.

mayeut · 2024-07-20T08:30:27Z

@timfel, this does not seem to work:

[root@78fc86164bc0 /]# uv -V
uv 0.2.27
[root@78fc86164bc0 /]# uv venv --python /opt/python/graalpy310-graalpy240_310_native/bin/python /tmp/uv-test-graalpy3.10
Using Python 3.10.13 interpreter at: opt/python/graalpy310-graalpy240_310_native/bin/python
Creating virtualenv at: tmp/uv-test-graalpy3.10
[root@78fc86164bc0 /]# /tmp/uv-test-graalpy3.10/bin/python -V
[To redirect Truffle log output to a file use one of the following options:
* '--log.file=<path>' if the option is passed using a guest language launcher.
* '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
* Configure logging using the polyglot embedding API.]
[python::PythonContext] WARNING: could not determine Graal.Python's standard library path. You need to pass --python.StdLibHome if you want to use the standard library.
[python::PythonContext] WARNING: could not determine Graal.Python's core path - you may need to pass --python.CoreHome.
[python::PythonContext] WARNING: could not determine Graal.Python's C API library path. You need to pass --python.CAPI if you want to use the C extension modules.
[python::PythonContext] WARNING: could not determine Graal.Python's C API library path. You need to pass --python.CAPI if you want to use the C extension modules.
ERROR: java.lang.UnsupportedOperationException: Could not load posix support library from path '/lib/graalpy24.0/libposix.so'. Troubleshooting: 
Check permissions of the file.
Missing runtime Maven dependency 'org.graalvm.truffle:truffle-nfi-libffi' (should be a dependency of `org.graalvm.polyglot:python{-community}`)?
org.graalvm.polyglot.PolyglotException: java.lang.UnsupportedOperationException: Could not load posix support library from path '/lib/graalpy24.0/libposix.so'. Troubleshooting: 
Check permissions of the file.
Missing runtime Maven dependency 'org.graalvm.truffle:truffle-nfi-libffi' (should be a dependency of `org.graalvm.polyglot:python{-community}`)?
	at com.oracle.graal.python.runtime.NFIPosixSupport$InvokeNativeFunction.loadLibrary(NFIPosixSupport.java:394)
	at com.oracle.graal.python.runtime.NFIPosixSupport$InvokeNativeFunction.call(NFIPosixSupport.java:330)
	at com.oracle.graal.python.runtime.NFIPosixSupport$InvokeNativeFunction.callLong(NFIPosixSupport.java:345)
	at com.oracle.graal.python.runtime.NFIPosixSupport.lseek(NFIPosixSupport.java:653)
	at com.oracle.graal.python.runtime.NFIPosixSupportGen$PosixSupportLibraryExports$Uncached.lseek(NFIPosixSupportGen.java:6569)
	at com.oracle.graal.python.runtime.ImageBuildtimePosixSupport.lseek(ImageBuildtimePosixSupport.java:261)
	at com.oracle.graal.python.runtime.ImageBuildtimePosixSupportGen$PosixSupportLibraryExports$Uncached.lseek(ImageBuildtimePosixSupportGen.java:2575)
	at com.oracle.graal.python.runtime.PosixSupportLibraryGen$UncachedDispatch.lseek(PosixSupportLibraryGen.java:7580)
	at com.oracle.graal.python.builtins.modules.io.FileIOBuiltins$SeekNode.internalSeek(FileIOBuiltins.java:812)
	at com.oracle.graal.python.builtins.modules.io.FileIOBuiltins$TellNode.internalTell(FileIOBuiltins.java:837)
	at com.oracle.graal.python.builtins.modules.io.AbstractBufferedIOBuiltins$BufferedInitNode.internalInit(AbstractBufferedIOBuiltins.java:120)
	at com.oracle.graal.python.builtins.modules.io.BufferedReaderBuiltins$BufferedReaderInit.internalInit(BufferedReaderBuiltins.java:103)
	at com.oracle.graal.python.builtins.modules.SysModuleBuiltins.initStd(SysModuleBuiltins.java:726)
	at com.oracle.graal.python.builtins.modules.SysModuleBuiltins.postInitialize(SysModuleBuiltins.java:710)
	at com.oracle.graal.python.builtins.Python3Core.postInitialize(Python3Core.java:980)
	at com.oracle.graal.python.runtime.PythonContext.patch(PythonContext.java:1485)
	at com.oracle.graal.python.PythonLanguage.patchContext(PythonLanguage.java:420)
	at com.oracle.graal.python.PythonLanguage.patchContext(PythonLanguage.java:142)
	at org.graalvm.truffle/com.oracle.truffle.api.LanguageAccessor$LanguageImpl.patchEnvContext(LanguageAccessor.java:431)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotLanguageContext.patch(PolyglotLanguageContext.java:893)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.patch(PolyglotContextImpl.java:3490)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineImpl.loadPreinitializedContext(PolyglotEngineImpl.java:1929)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineImpl.createContext(PolyglotEngineImpl.java:1809)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineDispatch.createContext(PolyglotEngineDispatch.java:152)
	at org.graalvm.polyglot/org.graalvm.polyglot.Context$Builder.build(Context.java:1937)
	at com.oracle.graal.python.shell.GraalPythonMain.launch(GraalPythonMain.java:739)
	at com.oracle.graal.python.enterprise.shell.GraalPythonEnterpriseMain.launch(GraalPythonEnterpriseMain.java:53)
	at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:296)
	at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:121)
	at org.graalvm.launcher.AbstractLanguageLauncher.runLauncher(AbstractLanguageLauncher.java:168)
	Suppressed: Attached Guest Language Frames (0)
Internal GraalVM error, please report at https://github.com/oracle/graal/issues/.
[root@78fc86164bc0 /]# /opt/python/graalpy310-graalpy240_310_native/bin/python -m venv /tmp/venv-test-graalpy3.10
[root@78fc86164bc0 /]# /tmp/venv-test-graalpy3.10/bin/python -V
GraalPy 3.10.13 (Oracle GraalVM Native 24.0.2)
[root@78fc86164bc0 /]# cat /tmp/uv-test-graalpy3.10/pyvenv.cfg 
home = /opt/_internal/graalpy310-graalpy240_310_native/bin
implementation = GraalVM
uv = 0.2.27
version_info = 3.10.13
include-system-site-packages = false
[root@78fc86164bc0 /]# cat /tmp/venv-test-graalpy3.10/pyvenv.cfg 
home = /opt/_internal/graalpy310-graalpy240_310_native
include-system-site-packages = false
version = 3.10.13

Removing /bin in pyvenv.cfg works but seems like a workaround specific to GraalPy ?

timfel · 2024-07-20T08:54:32Z

@timfel, this does not seem to work:

Yeah, there's a workaround in the CI config I commented on above in this PR, the fix is in GraalPy master and will be in the next release in September. Until then we have to work around with either your suggestion or setting GRAAL_PYTHONHOME like I do in the CI here

timfel mentioned this pull request Jul 17, 2024

Support GraalPy pypa/cibuildwheel#1538

Merged

timfel force-pushed the tim/graalpy branch from 82b7446 to 7bb3ef8 Compare July 17, 2024 11:21

zanieb added the enhancement New feature or improvement to existing functionality label Jul 17, 2024

zanieb self-assigned this Jul 17, 2024

zanieb reviewed Jul 17, 2024

View reviewed changes

crates/uv-virtualenv/src/bare.rs Show resolved Hide resolved

timfel force-pushed the tim/graalpy branch 2 times, most recently from edd6e5f to ff3dd22 Compare July 18, 2024 07:57

timfel commented Jul 18, 2024

View reviewed changes

zanieb mentioned this pull request Jul 18, 2024

Add script for integration tests #5189

Open

zanieb approved these changes Jul 18, 2024

View reviewed changes

timfel added 2 commits July 18, 2024 22:25

Add GraalPy support

a28d319

Work around GraalPy 24.0 bug with home finding (fixed in 24.1)

c954c6f

timfel force-pushed the tim/graalpy branch from ff3dd22 to c954c6f Compare July 18, 2024 20:25

zanieb merged commit 24a0268 into astral-sh:main Jul 19, 2024

timfel deleted the tim/graalpy branch July 19, 2024 03:31

mkniewallner mentioned this pull request Jul 19, 2024

GraalPy support #4197

Closed

BrewTestBot mentioned this pull request Jul 20, 2024

uv 0.2.27 Homebrew/homebrew-core#177880

Merged

timfel mentioned this pull request Oct 10, 2024

Python IDEs and Tools Compatibility oracle/graalpython#284

Open

masklinn mentioned this pull request Jan 1, 2025

Improve infrastructure ua-parser/uap-python#102

Open

12 tasks

paugier mentioned this pull request Apr 26, 2025

Support GraalPy install with uv python #13114

Open

masklinn mentioned this pull request Aug 2, 2025

Use uv to manage the project and handle CI setup ua-parser/uap-python#272

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add GraalPy support#5141

Add GraalPy support#5141
zanieb merged 2 commits intoastral-sh:mainfrom
timfel:tim/graalpy

timfel commented Jul 17, 2024 •

edited

Loading

Uh oh!

Uh oh!

zanieb commented Jul 17, 2024

Uh oh!

timfel commented Jul 18, 2024

Uh oh!

timfel Jul 18, 2024

Uh oh!

timfel Jul 18, 2024 •

edited

Loading

Uh oh!

zanieb commented Jul 18, 2024 •

edited

Loading

Uh oh!

zanieb left a comment

Uh oh!

mayeut commented Jul 20, 2024

Uh oh!

timfel commented Jul 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

timfel commented Jul 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

Uh oh!

zanieb commented Jul 17, 2024

Uh oh!

timfel commented Jul 18, 2024

Uh oh!

timfel Jul 18, 2024

Choose a reason for hiding this comment

Uh oh!

timfel Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanieb commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zanieb left a comment

Choose a reason for hiding this comment

Uh oh!

mayeut commented Jul 20, 2024

Uh oh!

timfel commented Jul 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

timfel commented Jul 17, 2024 •

edited

Loading

timfel Jul 18, 2024 •

edited

Loading

zanieb commented Jul 18, 2024 •

edited

Loading