Skip to content

ARROW-11259: [Python][Dataset] Allow creating nested field ref #9799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
pitrou wants to merge 1 commit into from
Closed

ARROW-11259: [Python][Dataset] Allow creating nested field ref #9799

pitrou wants to merge 1 commit into from

Conversation

@pitrou
Copy link
Member

@pitrou pitrou commented Mar 24, 2021

We can't yet filter on a nested field ref, so this merely adds syntactical support.

@pitrou
ARROW-11259: [Python][Dataset] Allow creating nested field ref
53c2bde 
We can't yet filter on a nested field ref, so this merely adds syntactical support.
bkietz
bkietz reviewed Mar 24, 2021
View reviewed changes
Copy link
Member

@bkietz bkietz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch seems well put together of itself, but does it add value to support construction of nested field references if we can't filter on them yet? It seems first we should refactor the parts of scan which depend on non nested field refs (ScanOptions::MaterializedFields, SetProjection) before exposing anything in python

@pitrou
Copy link
Member Author

pitrou commented Mar 24, 2021

but does it add value to support construction of nested field references if we can't filter on them yet

Probably not, but it also should be relatively orthogonal to future refactors?

@github-actions
Copy link

github-actions bot commented Mar 25, 2021

https://issues.apache.org/jira/browse/ARROW-11259

@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Mar 25, 2021

We do plan to support nested field references in filters? If so I am fine with already merging this, but no strong opinion.

@pitrou
Copy link
Member Author

pitrou commented Jun 23, 2021

Should I close this PR? @bkietz

@bkietz bkietz closed this Jun 28, 2021
@pitrou pitrou deleted the ARROW-11259-py-nested-field-ref branch August 26, 2021 12:03
@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Nov 10, 2021

Now there is actual support for nested field refs (#11466), this PR can be revived?

@pitrou
Copy link
Member Author

pitrou commented Dec 13, 2021

@jorisvandenbossche Do you want to take it up?

@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Dec 14, 2021

Yes, will do

@AlenkaF AlenkaF mentioned this pull request Apr 12, 2022
Closed
pitrou added a commit that referenced this pull request Apr 18, 2022
@AlenkaF @pitrou
ARROW-11259: [Python] Allow to create field reference to nested field
e0668e2 
This PR tries to redo the work from #9799.

It will unblock:
- https://issues.apache.org/jira/browse/ARROW-13798
- https://issues.apache.org/jira/browse/ARROW-14596

cc @jorisvandenbossche @pitrou

Closes #12863 from AlenkaF/ARROW-11259

Lead-authored-by: Alenka Frim <frim.alenka@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimport asfimport mentioned this pull request Apr 19, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bkietz bkietz bkietz left review comments

@jorisvandenbossche jorisvandenbossche Awaiting requested review from jorisvandenbossche

Assignees

No one assigned

Labels

Component: C++ Component: Python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pitrou @jorisvandenbossche @bkietz