fix: update string to search for in determing parquet column list separator#2670
Conversation
|
This distinction between Anyway, these are the only two names that need to be supported. |
Codecov Report
Additional details and impacted files
|
|
I am happy to merge this, unless @jpivarski has objections, I'll do so this evening. |
jpivarski
left a comment
There was a problem hiding this comment.
Now that I'm back at a computer, rather than a phone, I noticed that this is a PR on Awkward, not dask-awkward.
The change appears to be all-inclusive. If ".list.element" in column_metadata.path is needed to match new Arrow arrays and ".list.element." in column_metadata.path was used before, the new predicate will match both. Also, it couldn't spuriously match something it's not intended to.
So I sign off on it as well, and since we're all in agreement, I'll merge it, too.
…rame (#2735) * fix: backport numba old to new style error capturing to avoid warnings * backporting #2617 * backporting #2670 and tests that depend on Parquet list separator * style: pre-commit fixes * backporting #2617 for the _v2 subdirectory * RDataFrame handling is broken in _v2 subdirectory; don't bother fixing because it's correct in the real Awkward 2.x * The next awkward 1.x backport will be version 1.10.5. * Windows can't build ('x86' vs 'x86_64'). Maybe use the old VM? * No, change that back. * Remove all v2 tests. * I was changing the wrong workflow file. Let's try new Windows. * What I really need to test is the wheel-building. * Try again. * On second thought, don't upload to PyPI. * Remove 32-bit Windows from build-test.yml and modernize wheels.yml. --------- Co-authored-by: Jim Pivarski <jpivarski@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
With pyarrow 13 it appears that
list.elementis used overlist.itemin some datasets. Looks like our original check to look forlist.elementwas too strict (the final.). We also need to make this change in dask-awkward!more info at dask-contrib/dask-awkward#346