Skip to content

absolufy-imports - No relative imports - PEP8#7204

Merged
Illviljan merged 12 commits intopydata:mainfrom
Illviljan:absolufy_imports
Dec 7, 2022
Merged

absolufy-imports - No relative imports - PEP8#7204
Illviljan merged 12 commits intopydata:mainfrom
Illviljan:absolufy_imports

Conversation

@Illviljan
Copy link
Contributor

@Illviljan Illviljan commented Oct 24, 2022

I saw dask has started using absolute imports in dask/dask#8796.
I find it much more readable and there's a nice pre-commit for it as well.

Easiest way to deal with the merge conflicts is probably to just accept your changes and let pre-commit fix them afterwards.

@Illviljan Illviljan changed the title absolufy imports absolufy-imports - No relative imports - PEP8 Oct 24, 2022
@max-sixty
Copy link
Collaborator

I have no particular preference on which, but +0.5 on having a standard.

Also PyLance in VSCode has (temporarily, apparently) defaulted to making all imports absolute. So perhaps that will establish a standard.

@Illviljan Illviljan removed topic-performance topic-cftime topic-zarr Related to zarr storage library topic-arrays related to flexible array support topic-html-repr topic-rolling topic-combine combine/concat/merge run-benchmark Run the ASV benchmark workflow topic-typing io labels Oct 29, 2022
@headtr1ck headtr1ck added the plan to merge Final call for comments label Oct 30, 2022
@keewis
Copy link
Collaborator

keewis commented Oct 31, 2022

I'm a bit split on this.

As far as I understand it, there were two main reasons for the switch of dask / distributed: the fact that they had imports like

from .....scheduler import something

(which I agree is utter madness), and because they had modules named core, so the import was a bit ambiguous to read. I don't think either apply to us: our directory structure is sufficiently flat that we get at most two levels (from .. import something), and pretty unique file names.

We're also not in conflict with PEP8: if I read that correctly, it does say they recommend absolute imports, but then it also says that explicit relative imports are fine as well (and not just in the case where a complex structure makes explicit relative imports much more readable).

This means that unlike for dask one style does not have a clear advantage over the other, and this becomes a matter of preference.

As such, my vote would be +0.5 for automated consistency, but -0.5 for absolute imports everywhere because for me explicit relative imports are actually a bit easier to read. However, I very much agree with using absolute imports of modules from the main package in tests (I think we have something like from ..core.duck_array_ops import something in one of the tests, which seems a bit weird?)

@Illviljan
Copy link
Contributor Author

A lot of pep8 is quite gentle with the phrasing I think:
Avoid trailing whitespace anywhere. is only a recommendation.
Limit all lines to a maximum of 79 characters. is also only a recommendation, we ignore this one for automation consistency.

Another nice perk with absolute paths is that you can immediately run a utils.py file and test out the functions, much faster workflow than having to do the proper import route.

@github-actions github-actions bot added io run-benchmark Run the ASV benchmark workflow topic-arrays related to flexible array support labels Nov 7, 2022
@Illviljan
Copy link
Contributor Author

I am merging this on friday next week if no one minds. :)

@dcherian
Copy link
Contributor

Since it's a pretty 50/50 change lets discuss this at the next meeting on Dec 7.

(I bet changing the tests won't be controversial)

Copy link
Member

@shoyer shoyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this in the bi-weekly developers meeting and agreed to go forward, mostly for the sake of consistency with other projects and because it makes it easier to move code around.

@dcherian
Copy link
Contributor

dcherian commented Dec 7, 2022

Thanks for the contribution and the patience! @Illviljan

import xarray as xr

from . import parameterized, randn, requires_dask
from asv_bench.benchmarks import parameterized, randn, requires_dask
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes our benchmarks fail unfortunately:

    STDERR -------->
Error:    Traceback (most recent call last):
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 1435, in <module>
       main()
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 1428, in main
       commands[mode](args)
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 1103, in main_discover
       list_benchmarks(benchmark_dir, fp)
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 1088, in list_benchmarks
       for benchmark in disc_benchmarks(root):
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 985, in disc_benchmarks
       for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 967, in disc_modules
       for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
     File "/home/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/asv/benchmark.py", line 950, in disc_modules
       module = import_module(module_name)
     File "/home/runner/work/xarray/xarray/asv_bench/.asv/env/3e1d7de4e47af51e3e41695ae1884ae2/lib/python3.8/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
     File "<frozen importlib._bootstrap>", line 991, in _find_and_load
     File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 843, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/home/runner/work/xarray/xarray/asv_bench/benchmarks/dataarray_missing.py", line 4, in <module>
       from asv_bench.benchmarks import parameterized, randn, requires_dask
   ModuleNotFoundError: No module named 'asv_bench'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for using absolufy-imports - just for reference, in pandas we don't use it on asv_bench

https://github.com/pandas-dev/pandas/blob/4eb4729fb47bd160a39ec49f6af89dabfc63d3ac/.pre-commit-config.yaml#L8-L12

If you add a line to the hook with - files: ^xarray/, and revert the changes to asv_bench, then I think it should work fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Automation Github bots, testing workflows, release automation CI Continuous Integration tools plan to merge Final call for comments run-benchmark Run the ASV benchmark workflow topic-testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants