Skip to content

feat: Add Geometry & Geography Types #2859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
SaymV wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: Add Geometry & Geography Types #2859

SaymV wants to merge 4 commits into from
+1,060 −1

Conversation

@SaymV
Copy link

@SaymV SaymV commented Dec 23, 2025 •
edited
Loading

Closes #1820

Rationale for this change

Apache Iceberg v3 introduces native geometry and geography primitive types.
This PR adds spec-compliant support for those types in PyIceberg, including:

  • New primitive types with parameter handling and format-version enforcement
  • Schema parsing and round-trip serialization
  • Avro mapping using WKB bytes
  • PyArrow / Parquet integration with version-aware fallbacks
  • Unit test coverage for type behavior and compatibility constraints

A full design and scope discussion is available in the accompanying RFC:
📄 RFC: Iceberg v3 Geospatial Primitive Types
The RFC documents scope, non-goals, compatibility constraints, and known limitations.

Are these changes tested?

Yes.

  • Unit tests cover:
    • Type creation (default and custom parameters)
    • __str__ / __repr__ round-tripping
    • Equality, hashing, and pickling
    • Format version enforcement (v3-only)
  • PyArrow-dependent behavior is version-gated and conditionally tested

Are there any user-facing changes?

Yes.

  • Users may now declare geometry and geography columns in Iceberg v3 schemas
  • Parquet files written with PyArrow ≥ 21 preserve GEO logical types when available
  • Limitations (e.g. no WKB↔WKT conversion, no spatial predicates) are documented

SaymV added 4 commits December 22, 2025 14:32
@SaymV
feat: Add Iceberg v3 geospatial primitive types (geometry and geography)
9b36337 
Implement support for Iceberg v3 geospatial types as specified in the
Iceberg specification:

- Add GeometryType(crs) and GeographyType(crs, algorithm) to types.py
- Default CRS is "OGC:CRS84", default algorithm is "spherical"
- Types require format version 3 (minimum_format_version() returns 3)
- Values are stored as WKB (Well-Known Binary) bytes at runtime
- Avro schema conversion maps to "bytes"
- PyArrow conversion maps to large_binary()
- Add type string parsing for geometry('CRS') and geography('CRS', 'algo')
- Add visitor pattern support in schema.py and resolver.py

Note: JSON single-value encoding (WKB<->WKT) raises NotImplementedError
as it requires external libraries (e.g., Shapely) which are not included
to avoid heavy dependencies.
@SaymV
Add geoarrow dependency and document current capabilities
a7ac0d4
@SaymV
Add RFC
0d1d7a1
@SaymV
markdown lint fix
f4359b4
@SaymV SaymV changed the title Feat/geospatial types Add Geometry & Geography Types Dec 23, 2025
@SaymV SaymV changed the title Add Geometry & Geography Types feat: Add Geometry & Geography Types Dec 23, 2025
@Gawaboumga
Copy link

Gawaboumga commented Dec 26, 2025

Hopefully, I hope that the maintainers will be more reactive to integrate this PR than mine: #2224

@kevinjqliu
Copy link
Contributor

kevinjqliu commented Jan 16, 2026

Thanks for the PR! Great to see geoarrow is now available. I'm not too familiar with the Geo part of the Iceberg spec, so let me go read that first before reviewing this PR.

This PR already look pretty modular but if there's any way to break the PR into components, that would be very helpful.

Fokko
Fokko approved these changes Jan 16, 2026
View reviewed changes
Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super familiar with geo support, but from a PyIceberg perspective this looks like a great first start 👍

@SaymV
Copy link
Author

SaymV commented Jan 16, 2026

Thank you for the reviews! This PR is the groundwork.

The next phase, building on the existing GeometryType/GeographyType foundation, will be:

  1. Integration Tests & Round-Trip - End-to-end verification
  2. Bounds Metrics - WKB parsing for query optimization.
  3. Spatial Predicate Helpers - Expressions to use the stored bounds metrics (ST_Contains, ST_Intersects, ST_Within, ST_Overlaps)

I'll be sure to gate the dependencies with the modularity [geospatial].

My interest in this functionality is mostly personal, I've been playing around with exported Google Timeline data (that they provide in json via Google Takeout and would like to set something up for regular export and storage in a much more performant and stable format.

Happy to change/move/remove the documentation I supplied also. I felt like the RFC was necessary to include for discussion, but I'm not sure about the team's desire to keep those docs in the repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Fokko Fokko Fokko approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add GeometryType / GeographyType

4 participants

@SaymV @Gawaboumga @kevinjqliu @Fokko