Skip to content

Add example using pandas Series to map_partition doctring (#7323)#9161

Merged
jsignell merged 2 commits intodask:mainfrom
alex-gregory-ds:7323-series-map-partition-example
Jun 7, 2022
Merged

Add example using pandas Series to map_partition doctring (#7323)#9161
jsignell merged 2 commits intodask:mainfrom
alex-gregory-ds:7323-series-map-partition-example

Conversation

@alex-gregory-ds
Copy link
Copy Markdown
Contributor

@alex-gregory-ds alex-gregory-ds commented Jun 3, 2022 •
edited
Loading

@GPUtester
Copy link
Copy Markdown
Collaborator

GPUtester commented Jun 3, 2022

Can one of the admins verify this patch?

pavithraes
pavithraes reviewed Jun 3, 2022
View reviewed changes
Comment thread dask/dataframe/core.py Outdated

>>> d = [1, 2, 3, 4, 5]
>>> s = pd.Series(data=d)
>>> ds = dd.from_pandas(s, npartitions=2)
Copy link
Copy Markdown
Member

@pavithraes pavithraes Jun 3, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a part of the border example, I'm wondering if it'll be nicer to re-use the same ddf in the example instead of defining a new Series?

Something like:

>>> res = ddf.x.map_partitions(lambda x: len(x)) # ddf.x is a Dask Series Structure
>>> res.dtype
dtype('int64')

Copy link
Copy Markdown
Contributor Author

@alex-gregory-ds alex-gregory-ds Jun 3, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes a lot of sense, thanks for the suggestion! Added in commit 9595b03.

@pavithraes
Copy link
Copy Markdown
Member

pavithraes commented Jun 3, 2022

@Alex-JG3 I see this is your first PR. Welcome to Dask, and thanks for opening this!

@jakirkham
Copy link
Copy Markdown
Member

jakirkham commented Jun 3, 2022

Nothing for the PR author to do here (this is a note about a flaky CI test).

This CI build failed on Linux Python 3.10 with test_blockwise_dataframe_io[...-hdf]. IOW maybe related to issue ( #8816 ). The CI was run on this merge commit ( 35d0b42 ), which included this mitigation ( #9154 )

cc @ian-r-rose (for vis)

@jakirkham jakirkham mentioned this pull request Jun 3, 2022
Open
@jsignell
Copy link
Copy Markdown
Member

jsignell commented Jun 7, 2022

ok to test

@jsignell
Copy link
Copy Markdown
Member

jsignell commented Jun 7, 2022

This looks great! Thanks for the contribution @Alex-JG3

@jsignell jsignell merged commit c5e4b7c into dask:main Jun 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pavithraes pavithraes pavithraes left review comments

Assignees

No one assigned

Labels

dataframe

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Series map_partition example in documentation

5 participants

@alex-gregory-ds @GPUtester @pavithraes @jakirkham @jsignell