-
-
Notifications
You must be signed in to change notification settings - Fork 678
(bzlmod) Index responses containing relative paths for wheels causes bazel downloader to fail. #2150
Description
🐞 bug report
Affected Rule
The issue is caused by: _create_wheel_repos passing index-relative paths to whl_library instead of full URLs.
Is this a regression?
Probably not.
Description
The current implementation of whl_library passes URLs parsed from simpleapi_download verbatim to rctx.download. However, some indexes (like AWS CodeArtifact) return relative URLs. This causes the bazel downloader to choke when experimental_index_url is set to an index that returns relative URLs.
Expand to show GET response for private index vs pypi
curl --basic -u "aws:$auth_token" 'https://<redacted>.d.codeartifact.eu-central-1.amazonaws.com/pypi/packages-prod/simple/mypy-extensions/'
<!DOCTYPE html>
<html>
<head>
<title>Links for mypy-extensions</title>
</head>
<body>
<h1>Links for mypy-extensions</h1>
<a href="0.1.0/mypy_extensions-0.1.0.tar.gz#sha256=6942c0429e505a0cdcd0468fe897960fbd8113f84cb9bde1540120bc8b68eca9" data-gpg-sig="false" >mypy_extensions-0.1.0.tar.gz</a>
<br/>
<a href="0.2.0/mypy_extensions-0.2.0.tar.gz#sha256=b8f40b3bca27843f6ca911d63550799aa2586804d40da3ab087bab8130da3ebf" data-gpg-sig="false" >mypy_extensions-0.2.0.tar.gz</a>
<br/>
<a href="0.3.0/mypy_extensions-0.3.0.tar.gz#sha256=c4553f2475996c912658d3c2f398766cd47a65e0c2c6bb27ac8dea94a21640f4" data-gpg-sig="false" >mypy_extensions-0.3.0.tar.gz</a>
<br/>
<a href="0.4.0/mypy_extensions-0.4.0.tar.gz#sha256=118ec27adb66c682dc842bc2ad48d6d5218867df5f90ca713f4deb3760324fd3" data-gpg-sig="false" >mypy_extensions-0.4.0.tar.gz</a>
<br/>
<a href="0.4.1/mypy_extensions-0.4.1-py2.py3-none-any.whl#sha256=b16cabe759f55e3409a7d231ebd2841378fb0c27a5d1994719e340e4f429ac3e" data-gpg-sig="false" >mypy_extensions-0.4.1-py2.py3-none-any.whl</a>
<br/>
<a href="0.4.1/mypy_extensions-0.4.1.tar.gz#sha256=37e0e956f41369209a3d5f34580150bcacfabaa57b33a15c0b25f4b5725e0812" data-gpg-sig="false" >mypy_extensions-0.4.1.tar.gz</a>
<br/>
<a href="0.4.2/mypy_extensions-0.4.2-py2.py3-none-any.whl#sha256=c77e34350c01523cd63b6dc30b36515107b2c3cbacc8392d51d33ea18d59f8f1" data-gpg-sig="false" >mypy_extensions-0.4.2-py2.py3-none-any.whl</a>
<br/>
<a href="0.4.2/mypy_extensions-0.4.2.tar.gz#sha256=a161e3b917053de87dbe469987e173e49fb454eca10ef28b48b384538cc11458" data-gpg-sig="false" >mypy_extensions-0.4.2.tar.gz</a>
<br/>
<a href="0.4.3/mypy_extensions-0.4.3-py2.py3-none-any.whl#sha256=090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d" data-gpg-sig="false" >mypy_extensions-0.4.3-py2.py3-none-any.whl</a>
<br/>
<a href="0.4.3/mypy_extensions-0.4.3.tar.gz#sha256=2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8" data-gpg-sig="false" >mypy_extensions-0.4.3.tar.gz</a>
<br/>
<a href="0.4.4/mypy_extensions-0.4.4.tar.gz#sha256=c8b707883a96efe9b4bb3aaf0dcc07e7e217d7d8368eec4db4049ee9e142f4fd" data-requires-python=">=2.7" data-gpg-sig="false" >mypy_extensions-0.4.4.tar.gz</a>
<br/>
<a href="1.0.0/mypy_extensions-1.0.0-py3-none-any.whl#sha256=4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d" data-requires-python=">=3.5" data-gpg-sig="false" >mypy_extensions-1.0.0-py3-none-any.whl</a>
<br/>
<a href="1.0.0/mypy_extensions-1.0.0.tar.gz#sha256=75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782" data-requires-python=">=3.5" data-gpg-sig="false" >mypy_extensions-1.0.0.tar.gz</a>
<br/>
</body>
</html>curl 'https://pypi.org/simple/mypy-extensions/'
<!DOCTYPE html>
<html>
<head>
<meta name="pypi:repository-version" content="1.1">
<title>Links for mypy-extensions</title>
</head>
<body>
<h1>Links for mypy-extensions</h1>
<a href="https://files.pythonhosted.org/packages/77/9c/cad8e95677e005fc6d223f850e59a1751657a314e7f33aa06ffd5e209f28/mypy_extensions-0.1.0.tar.gz#sha256=6942c0429e505a0cdcd0468fe897960fbd8113f84cb9bde1540120bc8b68eca9" >mypy_extensions-0.1.0.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/aa/61/2c140b126809291baeb5642b626310c12373c0fa978fb7639ffeb5feaf5d/mypy_extensions-0.2.0.tar.gz#sha256=b8f40b3bca27843f6ca911d63550799aa2586804d40da3ab087bab8130da3ebf" >mypy_extensions-0.2.0.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/f7/51/a55d2c14e37821e6938b2e150afd50f0ed9ee4beaccb5d71b1014be63e6c/mypy_extensions-0.3.0.tar.gz#sha256=c4553f2475996c912658d3c2f398766cd47a65e0c2c6bb27ac8dea94a21640f4" >mypy_extensions-0.3.0.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/42/c8/3503632780b856dfd1ee4b3f6c61733833f9f58fe1a608dc8406054de99e/mypy_extensions-0.4.0.tar.gz#sha256=118ec27adb66c682dc842bc2ad48d6d5218867df5f90ca713f4deb3760324fd3" >mypy_extensions-0.4.0.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/4d/72/8d54e2b296631b9b14961d583e56e90d9d7fba8a240d5ce7f1113cc5e887/mypy_extensions-0.4.1-py2.py3-none-any.whl#sha256=b16cabe759f55e3409a7d231ebd2841378fb0c27a5d1994719e340e4f429ac3e" data-dist-info-metadata="sha256=15da51178c8af2bed1a429804dd2887b68ad35fdd404898e5dd429a99fa010e7" data-core-metadata="sha256=15da51178c8af2bed1a429804dd2887b68ad35fdd404898e5dd429a99fa010e7">mypy_extensions-0.4.1-py2.py3-none-any.whl</a><br />
<a href="https://files.pythonhosted.org/packages/c2/92/3cc05d1206237d54db7b2565a58080a909445330b4f90a6436302a49f0f8/mypy_extensions-0.4.1.tar.gz#sha256=37e0e956f41369209a3d5f34580150bcacfabaa57b33a15c0b25f4b5725e0812" >mypy_extensions-0.4.1.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/55/aa/2df16f9c6215b10c472fd7564b5f2c5644c13ab9c26d83b1c9373c11d97d/mypy_extensions-0.4.2-py2.py3-none-any.whl#sha256=c77e34350c01523cd63b6dc30b36515107b2c3cbacc8392d51d33ea18d59f8f1" data-dist-info-metadata="sha256=753214ffa231c23df663310f4d19d0d9fd75d729ff29de61335aa8b5cc3c1b5b" data-core-metadata="sha256=753214ffa231c23df663310f4d19d0d9fd75d729ff29de61335aa8b5cc3c1b5b">mypy_extensions-0.4.2-py2.py3-none-any.whl</a><br />
<a href="https://files.pythonhosted.org/packages/1e/2f/7bba47eb58f62a473387cd7658dedd0bedb4b0fa9d530bbbfa0a6d23034a/mypy_extensions-0.4.2.tar.gz#sha256=a161e3b917053de87dbe469987e173e49fb454eca10ef28b48b384538cc11458" >mypy_extensions-0.4.2.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/5c/eb/975c7c080f3223a5cdaff09612f3a5221e4ba534f7039db34c35d95fa6a5/mypy_extensions-0.4.3-py2.py3-none-any.whl#sha256=090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d" data-dist-info-metadata="sha256=e801077942edd68f705e69e548d4aedd0790731e18c2af0e2a107af3d486d29e" data-core-metadata="sha256=e801077942edd68f705e69e548d4aedd0790731e18c2af0e2a107af3d486d29e">mypy_extensions-0.4.3-py2.py3-none-any.whl</a><br />
<a href="https://files.pythonhosted.org/packages/63/60/0582ce2eaced55f65a4406fc97beba256de4b7a95a0034c6576458c6519f/mypy_extensions-0.4.3.tar.gz#sha256=2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8" >mypy_extensions-0.4.3.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/d6/c6/7809c27b5c5dacb9f6537e9489d969b266f7091204c75a03048edcb4daf9/mypy_extensions-0.4.4.tar.gz#sha256=c8b707883a96efe9b4bb3aaf0dcc07e7e217d7d8368eec4db4049ee9e142f4fd" data-requires-python=">=2.7" >mypy_extensions-0.4.4.tar.gz</a><br />
<a href="https://files.pythonhosted.org/packages/2a/e2/5d3f6ada4297caebe1a2add3b126fe800c96f56dbe5d1988a2cbe0b267aa/mypy_extensions-1.0.0-py3-none-any.whl#sha256=4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d" data-requires-python=">=3.5" data-dist-info-metadata="sha256=617d60dffb00826a22f5662114d8111d02fa29ffb1ec67e39d6814084ef75486" data-core-metadata="sha256=617d60dffb00826a22f5662114d8111d02fa29ffb1ec67e39d6814084ef75486">mypy_extensions-1.0.0-py3-none-any.whl</a><br />
<a href="https://files.pythonhosted.org/packages/98/a4/1ab47638b92648243faf97a5aeb6ea83059cc3624972ab6b8d2316078d3f/mypy_extensions-1.0.0.tar.gz#sha256=75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782" data-requires-python=">=3.5" >mypy_extensions-1.0.0.tar.gz</a><br />
</body>
</html>
<!--SERIAL 16697020-->🔬 Minimal Reproduction
MODULE.bazel
bazel_dep(name = "rules_python", version = "0.35.0")
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
is_default = True,
python_version = "3.11.5",
)
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
hub_name = "reqs",
experimental_index_url = "https://<redacted>.d.codeartifact.eu-central-1.amazonaws.com/pypi/packages-prod/simple",
python_version = "3.11.5",
requirements_lock = "//:requirements.txt",
)
use_repo(pip, "reqs")requirements.txt
mypy-extensions==1.0.0 \
--hash=sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d \
--hash=sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782
Running bazel fetch @reqs//mypy_extensions with the above files should reproduce the error below.
🔥 Exception or Error
ERROR: An error occurred during the fetch of repository 'rules_python~~pip~reqs_311_mypy_extensions_py3_none_any_4392f6c0':
Traceback (most recent call last):
File "/home/wmorrison/.cache/bazel/_bazel_wmorrison/0c20006a190ede26aadee6a87e8d620f/external/rules_python~/python/private/pypi/whl_library.bzl", line 216, column 31, in _whl_library_impl
result = rctx.download(
Error in download: java.io.IOException: Bad URL: 1.0.0/mypy_extensions-1.0.0-py3-none-any.whl
🌍 Your Environment
Operating System:
Ubuntu LinuxOutput of bazel version:
Bazelisk version: v1.20.0
Build label: 7.3.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Aug 19 16:12:50 2024 (1724083970)
Build timestamp: 1724083970
Build timestamp as int: 1724083970Rules_python version:
0.35.0Anything else relevant?
Related to #1357