Skip to content

python310Packages.aiohttp: add missing build inputs: setuptools, wheel#212602

Closed
ghost wants to merge 1 commit intostagingfrom
unknown repository
Closed

python310Packages.aiohttp: add missing build inputs: setuptools, wheel#212602
ghost wants to merge 1 commit intostagingfrom
unknown repository

Conversation

@ghost
Copy link
Copy Markdown

@ghost ghost commented Jan 25, 2023

Description of changes

Builds of aiohttpd have been failing on powerpc64le since 4553e62 with:

python3.10-aiohttp>   File "/nix/store/mhw4rs4xb20acvn55iw5i5k5rhaqidnq-python3-3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
python3.10-aiohttp>     return _bootstrap._gcd_import(name[level:], package, level)
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
python3.10-aiohttp> ModuleNotFoundError: No module named 'setuptools'
python3.10-aiohttp> note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-5'

This commit adds python3Packages.setuptools as a propagatedBuildInput, which many other python packages seem to do. It fixes the build. I don't know much python, so if there is a better/preferred way to address this please let me know.

Things done

@ghost ghost requested review from FRidh and jonringer as code owners January 25, 2023 12:45
@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 25, 2023
@ofborg ofborg bot requested a review from dotlambda January 25, 2023 12:58
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 1001-2500 This PR causes many rebuilds on Darwin and should most likely target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. labels Jan 25, 2023
@ghost
Copy link
Copy Markdown
Author

ghost commented Jan 25, 2023

Let me know if this should go to staging instead. It isn't urgent.

@FRidh
Copy link
Copy Markdown
Member

FRidh commented Jan 25, 2023

Please share the full trace. It is not obvious here where it fails. I think it fails during a test and not the build but cannot be sure from this.

@ghost
Copy link
Copy Markdown
Author

ghost commented Jan 25, 2023

Please share the full trace.

python3.10-aiohttp> Sourcing python-remove-tests-dir-hook
python3.10-aiohttp> Sourcing python-catch-conflicts-hook.sh
python3.10-aiohttp> Sourcing python-remove-bin-bytecode-hook.sh
python3.10-aiohttp> Sourcing pip-build-hook
python3.10-aiohttp> Using pipBuildPhase
python3.10-aiohttp> Using pipShellHook
python3.10-aiohttp> Sourcing pip-install-hook
python3.10-aiohttp> Using pipInstallPhase
python3.10-aiohttp> Sourcing python-imports-check-hook.sh
python3.10-aiohttp> Using pythonImportsCheckPhase
python3.10-aiohttp> Sourcing python-namespaces-hook
python3.10-aiohttp> Sourcing python-catch-conflicts-hook.sh
python3.10-aiohttp> unpacking sources
python3.10-aiohttp> unpacking source archive /nix/store/i4j1w19y8nkdwjz0pa7491gzwyr3pydi-aiohttp-3.8.3.tar.gz
python3.10-aiohttp> source root is aiohttp-3.8.3
python3.10-aiohttp> setting SOURCE_DATE_EPOCH to timestamp 1663765788 of file aiohttp-3.8.3/setup.cfg
python3.10-aiohttp> patching sources
python3.10-aiohttp> updateAutotoolsGnuConfigScriptsPhase
python3.10-aiohttp> configuring
python3.10-aiohttp> no configure script, doing nothing
python3.10-aiohttp> building
python3.10-aiohttp> Executing pipBuildPhase
python3.10-aiohttp> Creating a wheel...
python3.10-aiohttp> WARNING: The directory '/homeless-shelter/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
python3.10-aiohttp> Processing /build/aiohttp-3.8.3
python3.10-aiohttp>   Running command Preparing metadata (pyproject.toml)
python3.10-aiohttp>   Preparing metadata (pyproject.toml) ... done
python3.10-aiohttp> ERROR: Exception:
python3.10-aiohttp> Traceback (most recent call last):
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
python3.10-aiohttp>     status = run_func(*args)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
python3.10-aiohttp>     return func(self, options, args)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/commands/wheel.py", line 170, in run
python3.10-aiohttp>     requirement_set = resolver.resolve(reqs, check_supported_wheels=True)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
python3.10-aiohttp>     collected = self.factory.collect_root_requirements(root_reqs)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
python3.10-aiohttp>     req = self._make_requirement_from_install_req(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
python3.10-aiohttp>     cand = self._make_candidate_from_link(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
python3.10-aiohttp>     self._link_candidate_cache[link] = LinkCandidate(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
python3.10-aiohttp>     super().__init__(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
python3.10-aiohttp>     self.dist = self._prepare()
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
python3.10-aiohttp>     dist = self._prepare_distribution()
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
python3.10-aiohttp>     return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
python3.10-aiohttp>     return self._prepare_linked_requirement(req, parallel_builds)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
python3.10-aiohttp>     dist = _get_prepared_distribution(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
python3.10-aiohttp>     abstract_dist.prepare_distribution_metadata(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
python3.10-aiohttp>     self.req.prepare_metadata()
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 535, in prepare_metadata
python3.10-aiohttp>     self.metadata_directory = generate_metadata(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
python3.10-aiohttp>     distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 706, in prepare_metadata_for_build_wheel
python3.10-aiohttp>     return super().prepare_metadata_for_build_wheel(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 189, in prepare_metadata_for_build_wheel
python3.10-aiohttp>     return self._call_hook('prepare_metadata_for_build_wheel', {
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 319, in _call_hook
python3.10-aiohttp>     raise BackendUnavailable(data.get('traceback', ''))
python3.10-aiohttp> pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 77, in _build_backend
python3.10-aiohttp>     obj = import_module(mod_path)
python3.10-aiohttp>   File "/nix/store/mhw4rs4xb20acvn55iw5i5k5rhaqidnq-python3-3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
python3.10-aiohttp>     return _bootstrap._gcd_import(name[level:], package, level)
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
python3.10-aiohttp> ModuleNotFoundError: No module named 'setuptools'
python3.10-aiohttp> note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-7'
error: build of '/nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv' on 'ssh://nix@stallman' failed: error: builder for '/nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv' failed with exit code 2;
              last 10 log lines:
              >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
              >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
              >   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
              >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
              >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
              >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
              >   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
              > ModuleNotFoundError: No module named 'setuptools'
              >
              >
              For full logs, run 'nix log /nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv'.
error: builder for '/nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv' failed with exit code 1;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
       >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
       > ModuleNotFoundError: No module named 'setuptools'
       >
       >
       > note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-7'
       For full logs, run 'nix log /nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv'.

@FRidh
Copy link
Copy Markdown
Member

FRidh commented Jan 25, 2023

Thanks. Checking the trace, it seems to fail during wheel building. Looking at the expression, it seems the build system is not added. It does manage to get access to it via the runtime dependencies (propagatedBuildInputs), at least on other platforms, but that's not how it is supposed to be.

You can add instead of your current fix

nativeBuildInputs = [
  setuptools
  wheel
];

to solve this.

Also please follow the contribution guidelines when it comes to the commit title.

@dotlambda dotlambda changed the title python-modules/aiohttpd: add missing setuptools buildinput python310Packages.aiohttpd: add missing setuptools build input Jan 26, 2023
@dotlambda dotlambda changed the title python310Packages.aiohttpd: add missing setuptools build input python310Packages.aiohttp: add missing setuptools build input Jan 26, 2023
Copy link
Copy Markdown
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please follow the contribution guidelines when it comes to the commit title.

See PR title.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FRidh Why wheel? It's not specified in upstream's pyproject.toml.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wheel package is always needed to make a wheel. Maybe we get away with it because pip that we instruct to build the wheel brings it in. I don't know anymore.

@ghost ghost changed the title python310Packages.aiohttp: add missing setuptools build input aiohttp: add missing build inputs: setuptools, wheel Jan 26, 2023
@ghost ghost requested review from dotlambda and removed request for FRidh and jonringer January 26, 2023 12:52
Copy link
Copy Markdown
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit message should start with e.g. python310Packages.. Please target staging.

Builds of aiohttpd have been failing on powerpc64le since
4553e62 with:

```
python3.10-aiohttp>   File "/nix/store/mhw4rs4xb20acvn55iw5i5k5rhaqidnq-python3-3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
python3.10-aiohttp>     return _bootstrap._gcd_import(name[level:], package, level)
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
python3.10-aiohttp> ModuleNotFoundError: No module named 'setuptools'
python3.10-aiohttp> note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-5'
```

This commit adds `setuptools` and `wheel` as `nativeBuildInputs` to
fix this.
@ghost ghost changed the title aiohttp: add missing build inputs: setuptools, wheel python310Packages.aiohttp: add missing build inputs: setuptools, wheel Jan 27, 2023
@ghost ghost marked this pull request as draft January 27, 2023 02:25
@ghost ghost changed the base branch from master to staging January 27, 2023 02:26
@ghost ghost marked this pull request as ready for review January 27, 2023 02:26
@ghost ghost requested a review from dotlambda January 27, 2023 02:26
@ofborg ofborg bot added 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. and removed 10.rebuild-linux: 2501-5000 This PR causes many rebuilds on Linux and should target the staging branches. labels Jan 27, 2023
Copy link
Copy Markdown
Contributor

@lopsided98 lopsided98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that this fixes cross-compiling to armv6l-linux. The build still succeeds without wheel, but I don't know why it should or shouldn't be included.

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 24, 2023

This can wait until after ZHF.

@ghost ghost marked this pull request as draft April 24, 2023 05:27
Copy link
Copy Markdown
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not tested but I don't see any reason why this would break anything.

@FRidh
Copy link
Copy Markdown
Member

FRidh commented Aug 11, 2023

Fixed with #247310

@FRidh FRidh closed this Aug 11, 2023
@ghost ghost deleted the pr/python/aiohttpd/fix branch January 23, 2024 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 1001-2500 This PR causes many rebuilds on Darwin and should most likely target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants