Skip to content

Add deprecation warning for pyarow-legacy engine#8758

Merged
rjzamora merged 3 commits intodask:mainfrom
rjzamora:deprecate-pyarrow-legacy
Feb 25, 2022
Merged

Add deprecation warning for pyarow-legacy engine#8758
rjzamora merged 3 commits intodask:mainfrom
rjzamora:deprecate-pyarrow-legacy

Conversation

@rjzamora
Copy link
Copy Markdown
Member

@rjzamora rjzamora commented Feb 25, 2022

I'd like to work on some parquet cleanup and documentation, and dropping support for ArrowLegacyEngine will certainly simplify things. This PR adds a deprecation warning for engine="pyarrow-legacy".

I plan to submit a follow-up PR after 1-2 releases to remove ArrowLegacyEngine altogether.

@jakirkham
Copy link
Copy Markdown
Member

@martindurant would you have time to look? 🙂

Copy link
Copy Markdown
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
)

if engine == "pyarrow-dataset":
if pa_version.major < 1:
Copy link
Copy Markdown
Member

@martindurant martindurant Feb 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not require >1 by now?

Sorry, that's exactly what you are doing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably can since we dropped 3.7 support, but pyarrow-1 should still work with the pyarrow-dataset engine. I figured we could bump this later on when needed, but I'm open to anything

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What you have is fine. We'll simply remove the whole thing soon enough!

@martindurant
Copy link
Copy Markdown
Member

+1

@rjzamora rjzamora merged commit efa6b73 into dask:main Feb 25, 2022
@rjzamora rjzamora deleted the deprecate-pyarrow-legacy branch February 25, 2022 14:45
@rjzamora
Copy link
Copy Markdown
Member Author

Thanks @martindurant and @jorisvandenbossche for taking a look so quickly!

@jrbourbeau
Copy link
Copy Markdown
Member

Woo

@jakirkham
Copy link
Copy Markdown
Member

Thanks all! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants