Skip to content

Conversation

@gbrgr
Copy link
Collaborator

@gbrgr gbrgr commented Jan 8, 2026

No description provided.

gbrgr and others added 25 commits January 8, 2026 12:32
## Which issue does this PR close?

- Closes apache#1963.

## What changes are included in this PR?

This change validates that table metadata with reserved sort order ID
(0) cannot contain fields associated with it. If this is found, we error
out instead of silently parsing arbitrary field values.

## Are these changes tested?

Added the unit test described in the issue and verified that the check
is now enforced.
Implement LIMIT pushdown to optimize queries with LIMIT clauses by
stopping data processing once the limit is reached. This reduces
unnecessary I/O and computation for queries that only need a subset of
rows.

Changes:
- Add limit field to IcebergTableScan to track row limit
- Apply limit at stream level by filtering/slicing record batches
- Update IcebergTableProvider and IcebergStaticTableProvider to pass
limit parameter to scan
- Add comprehensive tests for limit pushdown functionality

## Which issue does this PR close?


- Closes #.

## What changes are included in this PR?


## Are these changes tested?

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
@gbrgr gbrgr marked this pull request as ready for review January 9, 2026 08:31
@gbrgr gbrgr enabled auto-merge (squash) January 9, 2026 08:34
@gbrgr gbrgr disabled auto-merge January 9, 2026 08:34
@gbrgr gbrgr merged commit f022a2e into main Jan 9, 2026
16 of 25 checks passed
@gbrgr gbrgr deleted the gb/merge-upstream-arrow-57.1 branch January 9, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants