Skip to content

Pickling: fix SRID handling + update linearring test#1245

Merged
jorisvandenbossche merged 5 commits intoshapely:mainfrom
jorisvandenbossche:pickling
Feb 11, 2022
Merged

Pickling: fix SRID handling + update linearring test#1245
jorisvandenbossche merged 5 commits intoshapely:mainfrom
jorisvandenbossche:pickling

Conversation

@jorisvandenbossche
Copy link
Copy Markdown
Member

@jorisvandenbossche jorisvandenbossche commented Dec 2, 2021

The goal here is to consolidate the pickling behaviour between pygeos and shapely. The differences:

  • Shapely preserved LinearRing in the roundtrip, pygeos not (some discussion about this in Pickling of LinearRings #988)
  • Pygeos preserved the SRID in roundtrip, shapely not

This PR combines both features, preserving both LinearRing type and SRID.

I currently did this in the Python layer, while we also have a base __reduce__ implementation in C (which is probably unused now in practice):

https://github.com/Toblerity/Shapely/blob/2e513b143c1f0a84b9cd23cbe409f72bfa78f40f/src/pygeom.c#L408

I suppose we could also keep the logic in C. The main reason that I currently need to override __reduce__ in the python subclasses is because the C version ends up passing the WKB to the class __new__. The base lib.Geometry class accepts WKB for construction, but the subclasses currently do not (eg shapely.geometry.Point(..) only accepts actual coordinate values).

Closes #988

@caspervdw
Copy link
Copy Markdown
Collaborator

In pygeos, we didn't have the Geometry subclasses, so things like pickling were implemented in C. I am really 👍 on keeping that kind of per-object logic on the Python side.

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

I suppose I can then actually remove the C version (GeometryObject_reduce)

@sgillies
Copy link
Copy Markdown
Contributor

sgillies commented Dec 4, 2021

Looks good. I still don't like the idea of SRID support. Not supporting SRID up to this point has eliminated a source of complexity and thus has been good for Shapely. And the GEOS project doesn't consider SRID to be a useful feature, either.

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

As long as we limit the SRID support strictly to IO (i.e. pickling and read/write WKB), I don't think it will really add a source of complexitly / issues. If someone opens an issue related to SRID for anything else, we can simply close as out-of-scope / defer to GEOS.

I personally don't care that much about preserving SRID for pickling, as long as we support it for WKB (which is needed for example for PostGIS interaction)

@caspervdw
Copy link
Copy Markdown
Collaborator

Personally I also think that srids are out of scope. However if users want to carry this information (any information really) along with the geometries, then I think it is fine to support that. As long as we never base any logic on it.

@sgillies
Copy link
Copy Markdown
Contributor

sgillies commented Dec 7, 2021

I looked in geopandas and it seems that it is only depending on a geometry's SRID due to a design choice and not because there is no other way. Instead of checking to see if an SRID is set on the first geometry at https://github.com/geopandas/geopandas/blob/d006fad416a9f33fd88c1a00744ae9f98cb9c6d0/geopandas/io/sql.py#L88 (if there was a bug here it would be super confusing), this function could instead call https://postgis.net/docs/Find_SRID.html via pd.read_sql() or some lower level function.

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

We still use SRID for WKB export to PostGIS as well (that doesn't need to be on the geometry though, but as mentioned above still need it in the WKB IO functions)
(this relates to pygeos/pygeos#137)

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

For the reading side, it is true that we don't need the get_srid(geom) for PostGIS, as we could indeed use one of the PostGIS function as well. However, I personally think we should still support reading the SRID from a WKB object. You can have other sources of WKB than PostGIS, and I think generally being able to extract all information contained in the WKB is a good thing to support.

@coveralls
Copy link
Copy Markdown

coveralls commented Dec 27, 2021

Pull Request Test Coverage Report for Build 1627350912

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 7 of 7 (100.0%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+2.9%) to 83.969%

Totals Coverage Status
Change from base Build 1627324933: 2.9%
Covered Lines: 2200
Relevant Lines: 2620

💛 - Coveralls

@jorisvandenbossche
Copy link
Copy Markdown
Member Author

Since this PR is only fixing the LinearRing pickling and making it consistent with the other geometry classes, I am planning to merge this.
We can continue the discussion about to what extent we want to expose SRID related functionality in general in another issue.

@jorisvandenbossche jorisvandenbossche merged commit 02a5b6e into shapely:main Feb 11, 2022
@jorisvandenbossche jorisvandenbossche deleted the pickling branch February 11, 2022 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pickling of LinearRings

4 participants