-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
highlevelgraphIssues relating to HighLevelGraphs.Issues relating to HighLevelGraphs.
Description
Blocked by #6715
Blocked by #7281
After #7281 goes in, most (maybe all) IO operations in the dataframe and array modules can be converted into Blockwise high-level graph (HLG) layers. This change should improve HLG optimizations (like blockwise fusion).
This issue is intended as a "living" checklist of action items related to Blockwise IO...
Dask-Dataframe:
- read_parquet [Use Blockwise for DataFrame IO (parquet, csv, and orc) #7415]
- read_csv (via text_blocks_to_pandas) [Use Blockwise for DataFrame IO (parquet, csv, and orc) #7415]
- read_orc [Use Blockwise for DataFrame IO (parquet, csv, and orc) #7415]
- make_timeseries [Move timeseries and daily-stock to Blockwise #7615]
- daily_stock [Move timeseries and daily-stock to Blockwise #7615]
- read_hdf [Move read_hdf to Blockwise #7625]
- from_array
- from_pandas
- from_bcolz
- from_dask_array
- from_delayed
- read_json
Dask-Array:
- from_array
- from_zarr (may be covered by
from_array) - from_delayed
- from_func
- empty_like
- ones_like [Prepare Blockwise + IO infrastructure #7281]
- zeros_like [Prepare Blockwise + IO infrastructure #7281]
- full_like [Prepare Blockwise + IO infrastructure #7281]
- linspace
- arange
- eye
- diag
- diagonal
- triu
- tril
- fromfunction [Rewrite
da.fromfunctionw/da.blockwise#7704] - repeat
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
highlevelgraphIssues relating to HighLevelGraphs.Issues relating to HighLevelGraphs.