Skip to content

Update tz for pandas 1.4.0#8626

Merged
jsignell merged 2 commits intodask:mainfrom
martindurant:parquet_tz
Jan 27, 2022
Merged

Update tz for pandas 1.4.0#8626
jsignell merged 2 commits intodask:mainfrom
martindurant:parquet_tz

Conversation

@martindurant
Copy link
Member

Also requires dask/fastparquet#738

@jrbourbeau
Copy link
Member

Just pushed an empty commit to test against the main branch of fastparquet (given dask/fastparquet#738 has been merged). Hope that's okay

@martindurant
Copy link
Member Author

Yep. I thought that restarting the tests would have the same effect, but maybe not.

@martindurant
Copy link
Member Author

This does not seem to be picking up fastparquet main in every run. Do I need to make a release? The (parquet) tests do all pass for me locally.

@jsignell
Copy link
Member

Only the upstream test will pick up fastparquet from main. So yeah I think a release would be necessary to get the regular tests passing.

@martindurant
Copy link
Member Author

OK; well upstream seems to be failing on loads of dataframe tests (from warnings?). They include some parquet, but I cannot tell if these are actually parquet failures or due to whatever broke elsewhere.

@jrbourbeau
Copy link
Member

Comparing the failing test summary between the linux Python 3.9 build (which is using pandas=1.4.0) and the upstream build, the changes in this PR + the fastparquet fixes result in dask/dataframe/io/tests/test_parquet.py::test_roundtrip[fastparquet*] and dask/dataframe/io/tests/test_parquet.py::test_timestamp96 passing. #8623 should address the other failures on pandas=1.4.0

@martindurant
Copy link
Member Author

OK, so we merge and I'll release fastparquet? There are a couple of PRs waiting there that would be nice to get in first.

@jrbourbeau jrbourbeau mentioned this pull request Jan 26, 2022
3 tasks
@jrbourbeau
Copy link
Member

A fastparquet release would be great. Do you have a rough estimate on how long that might take? It would be good to have the Python 3.9 builds pass here, so if it will still be a bit before those other PRs can get in, then we might consider using the dev version of fastparquet in our 3.9 builds

@martindurant
Copy link
Member Author

Coming... dask/fastparquet#739

@jrbourbeau
Copy link
Member

Woo 🎉

@jsignell
Copy link
Member

Ok I am going to merge this and then we should be all green???? :)

@jsignell jsignell merged commit 193998c into dask:main Jan 27, 2022
@martindurant martindurant deleted the parquet_tz branch January 27, 2022 14:36
@jsignell
Copy link
Member

Thank you @martindurant for the quick turn around on this!

@martindurant
Copy link
Member Author

Hope! Note that conda-forge did it's job, which is what matters here, but some pypi wheels didn't build. I'll track those down later.

@jrbourbeau
Copy link
Member

Thanks @martindurant @jsignell!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failures with upcoming pandas 1.4 release

3 participants