Skip to content

DataFrame.head shouldn't warn when there's one partition #8088

@TomAugspurger

Description

@TomAugspurger

dask.data

In [12]: import dask.dataframe as dd, pandas as pd

In [13]: df = dd.from_pandas(pd.DataFrame({"A": [1, 2]}), npartitions=1)

In [14]: df.head()
/home/taugspurger/miniconda3/envs/stac-table/lib/python3.9/site-packages/dask/dataframe/core.py:6778: UserWarning: Insufficient elements for `head`. 5 elements requested, only 2 elements available. Try passing larger `npartitions` to `head`.
  warnings.warn(msg.format(n, len(r)))
Out[14]:
   A
0  1
1  2

When there's just a single partition (or, more generally, if all the partitions are used), then there's no reason to warn.

I think the fix is change

dask/dask/dataframe/core.py

Lines 1081 to 1084 in 7cfee4a

if safe:
head = safe_head
else:
head = M.head
or perhaps
return self._head(n=n, npartitions=npartitions, compute=compute, safe=True)
to use safe=False when npartitions == self.npartitions (or when npartitions=-1).

Metadata

Metadata

Assignees

No one assigned

    Labels

    dataframegood first issueClearly described and easy to accomplish. Good for beginners to the project.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions