TST: assert reading of legacy pickles against current data#61792
TST: assert reading of legacy pickles against current data#61792jorisvandenbossche merged 9 commits intopandas-dev:mainfrom
Conversation
| and legacy_version < Version("1.3.0") | ||
| ): | ||
| # convert to wall time | ||
| # (bug since pandas 2.0 that tz gets dropped for older pickle files) |
There was a problem hiding this comment.
is there an issue ref for this
There was a problem hiding this comment.
We had one: #54659, but reported for an even older pickle file, and therefore closed.
We can reopen that issue, but on the other hand, not sure this is still worth the time to fix since this already was broken the full 2.x cycle, and at some point in the future we will probably drop compat for 1.x pickles altogether. But of course if someone wants to do a PR, I suppose that is welcome.
|
can you merge main and see if the pyarrow decimal issue resolves itself? |
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
| for typ, dv in data.items(): | ||
| for dt, result in dv.items(): | ||
| expected = result | ||
| expected = current_data[typ][dt] |
There was a problem hiding this comment.
This is the actual fix to ensure we are testing things correctly (and not just comparing the result with itself)
|
Going to merge this to ensure we will be actually testing the pickle compat in 3.0.x branch |
While reviewing #61770, I noticed that we didn't actually compare the read pickle data to some ground truth expected value, but just to itself (we were essentially doing
assert_equal(result, result)..), due to some accidental change in a clean-up many years ago in f2246cf)Fixing that here by again creating the expected unpickled data with
create_pickle_data()during the test run, to compare with the data from the older pickled files.