Skip to content

Pandas 0.21 compatability#2737

Merged
TomAugspurger merged 5 commits intodask:masterfrom
TomAugspurger:pandas-021
Oct 5, 2017
Merged

Pandas 0.21 compatability#2737
TomAugspurger merged 5 commits intodask:masterfrom
TomAugspurger:pandas-021

Conversation

@TomAugspurger
Copy link
Copy Markdown
Member

@TomAugspurger TomAugspurger commented Oct 3, 2017

Each commit has the compatability change for a specific change in pandas.

This will fail for now because of pandas-dev/pandas#17767 and pandas-dev/pandas#16821. That will take me a little while longer to debug (short story: pandas is more consistent about dtype casting. I think pandas is doing the correct thing here.)

closes #2736

TomAugspurger
TomAugspurger commented Oct 3, 2017
View reviewed changes
dask/dataframe/io/tests/test_io.py
assert (sorted(dd.from_bcolz(t, chunksize=2).dask) !=
sorted(dd.from_bcolz(t, chunksize=3).dask))

threads = []
Copy link
Copy Markdown
Member Author

@TomAugspurger TomAugspurger Oct 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @mrocklin, this change was because the old style failed to ever raise an exception for me, even with

def check():
    assert 0

The exception would happen in the other thread, but would fail to cause an exception in the main test process.

@TomAugspurger
Copy link
Copy Markdown
Member Author

TomAugspurger commented Oct 3, 2017 •
edited
Loading

Nevermind, 466e51e won't work, since it breaks everything else :/ like np.array(scalar)

@TomAugspurger
Copy link
Copy Markdown
Member Author

TomAugspurger commented Oct 5, 2017

OK, this should be good on the dask side of things.

This worker was against pandas dev (https://travis-ci.org/dask/dask/jobs/283696522#L645) as of about 14 hours ago. Earlier this morning, pandas-dev/pandas#17779 was merged, which will make the only failing test pass.

pytest dask/dataframe/tests/test_arithmetics_reduction.py::test_scalar_arithmetics_with_dask_instances
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.6.1, pytest-3.2.2, py-1.4.34, pluggy-0.4.0
rootdir: /Users/taugspurger/Envs/pandas-dev/lib/python3.6/site-packages/dask, inifile:
plugins: xdist-1.15.0, rerunfailures-2.2, repeat-0.4.1, cov-2.5.1, hypothesis-3.26.0
collected 1 item

dask/dataframe/tests/test_arithmetics_reduction.py .

================================================================= 1 passed in 0.78 seconds =================================================================

At this point, I think I'll revert the change adding a pandas-dev worker, and just leave the compatibility changes?

@TomAugspurger TomAugspurger merged commit 3be1d97 into dask:master Oct 5, 2017
@TomAugspurger TomAugspurger deleted the pandas-021 branch October 5, 2017 18:52
@TomAugspurger
Copy link
Copy Markdown
Member Author

TomAugspurger commented Oct 5, 2017

@mrocklin should be good to go.

@mrocklin
Copy link
Copy Markdown
Member

mrocklin commented Oct 5, 2017 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

COMPAT: pandas 0.21.0

2 participants

@TomAugspurger @mrocklin