Skip to content

TST: fix tests on Windows for python >= 3.8#1213

Merged
jorisvandenbossche merged 10 commits intoshapely:shapely-2.0from
jorisvandenbossche:test-windows-py38
Oct 29, 2021
Merged

TST: fix tests on Windows for python >= 3.8#1213
jorisvandenbossche merged 10 commits intoshapely:shapely-2.0from
jorisvandenbossche:test-windows-py38

Conversation

@jorisvandenbossche
Copy link
Copy Markdown
Member

With pygeos installed, the tests on appveyor are failing for Python >= 3.8 (failing to import pygeos because it doesn't find GEOS). I assume this is related to the changes in python how it finds (or does not find) DLLs, and you need to explicitly put them on the path.

@coveralls
Copy link
Copy Markdown

coveralls commented Oct 28, 2021

Coverage Status

Coverage remained the same at 78.4% when pulling dcb22a2 on jorisvandenbossche:test-windows-py38 into 0af2ed3 on Toblerity:shapely-2.0.

@mwtoews
Copy link
Copy Markdown
Member

mwtoews commented Oct 28, 2021

See pygeos/pygeos#383

The best solution is to copy the DLLs, as the PATH fix doesn't work as found in pygeos/pygeos#382

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

Hmm, OK, I hoped that setting the path here would work, as copying the geos dlls into the pygeos directory is not that straightforward here, since we pip install pygeos (so I would need to find a robust way to get the path to the pygeos install location).

I also don't want to spend too much time on it, because after #1211, the next step would be to move the pygeos' repo content in here, and then one of the first steps would be to consolidate the CI scripts (and pygeos' scripts are already handling this fine). So temporarily disabling the windows tests for python >=3.8 (as I now did in #1211) is maybe also good enough

@mwtoews
Copy link
Copy Markdown
Member

mwtoews commented Oct 28, 2021

Maybe temporarily use os.add_dll_directory(path)? It would need to be embedded in the Python code.

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

I would do that in the testing code? (eg in conftest.py)

@mwtoews
Copy link
Copy Markdown
Member

mwtoews commented Oct 28, 2021

In theory it would need to be os.add_dll_directory(path) before the first import pygeos.

@mwtoews
Copy link
Copy Markdown
Member

mwtoews commented Oct 28, 2021

As the error stems from shapely\geos.py:4, it would need to be moved just before there. Otherwise the modification looks like it should work.

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

@mwtoews thanks for the help! This seems to be working, as there is now an actual failure:

    @requires_geos_380
    def test_point_empty():
        g = wkt.loads("POINT EMPTY")
>       assert g.wkb_hex == hostorder(
            "BIdd", "0101000000000000000000F87F000000000000F87F")
E       AssertionError: assert '010100000000...000000000F8FF' == '010100000000...000000000F87F'
E         - 0101000000000000000000F87F000000000000F87F
E         ?                         ^               -
E         + 0101000000000000000000F8FF000000000000F8FF
E         ?                         ^                +
tests\test_wkb.py:162: AssertionError

@mwtoews
Copy link
Copy Markdown
Member

mwtoews commented Oct 28, 2021

Empty strikes again! Looks familiar. PostGIS decodes both to POINT EMPTY:

postgis=# WITH data as (SELECT
postgis(#   '0101000000000000000000F87F000000000000F87F' A,
postgis(#   '0101000000000000000000F8FF000000000000F8FF' B)
postgis-# SELECT 'A'::text test, A, ST_AsText(A) FROM data
postgis-# UNION SELECT 'B',      B, ST_AsText(B) FROM data;
 test |                     a                      |  st_astext
------+--------------------------------------------+-------------
 A    | 0101000000000000000000F87F000000000000F87F | POINT EMPTY
 B    | 0101000000000000000000F8FF000000000000F8FF | POINT EMPTY
(2 rows)

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

Yes, both are valid representations (different kind of NaN values), but so when converting to WKB, which of the two you get depends on your architecture. What I added here makes that test more robust by asserting the bytes represent NaN, instead of hardcoding exactly the expected bytes (this is what we do in pygeos).

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

jorisvandenbossche commented Oct 28, 2021

This now includes the change I originally did in #1123

@mwtoews
Copy link
Copy Markdown
Member

mwtoews commented Oct 28, 2021

Last suggestion is to clean-up the "Checking geos_c.dll" section in appveyor.yml around line 55, as this was primarily for ctypes

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

Green! :)

appveyor.yml Outdated
#- python -c "import shapely; print(shapely.__version__)"
#- python -c "from shapely.geos import geos_version_string; print(geos_version_string)"
#- python -c "from shapely import speedups; assert speedups.enabled"
#- python -c "from shapely import vectorized"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe now try uncommenting the above lines?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think those simply can be removed. Both modules still exist, but are just a small shim not doing anything special for backwards compatibility (eg shapely.speedups just includes raising deprecation warnings). I don't think it is needed to specifically test those imports now.

Copy link
Copy Markdown
Member

@mwtoews mwtoews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@jorisvandenbossche jorisvandenbossche merged commit 2eed2d0 into shapely:shapely-2.0 Oct 29, 2021
@jorisvandenbossche jorisvandenbossche deleted the test-windows-py38 branch October 29, 2021 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants