-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
dataframeioneeds attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.It's been a while since this was pushed on. Needs attention from the owner or a maintainer.parquet
Description
Given the pyarrow version requirements in Dask, we can now assume the dataset API will be supported if pyarrow is installed. We can also assume that deprecation warnings will be raised by the pyarrow backend if dd.read_parquet(..., engine="pyarrow-legacy") is used. Therefore, I propose that we add an explicit deprecation warning for the "pyarrow-legacy" engine itself, and establish a timeline for its removal.
Deprecating and removing "pyarrow-legacy" should simplifyread_parquet maintenance, and should have few (if any) downsides. However, I welcome pushback if others expect this to cause pain or problems.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
dataframeioneeds attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.It's been a while since this was pushed on. Needs attention from the owner or a maintainer.parquet