Skip to content

ARROW-8922: [C++] Add illustrative "ascii_upper" and "ascii_length" scalar string functions valid for Array and Scalar inputs #7278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wesm wants to merge 3 commits into from
Closed

ARROW-8922: [C++] Add illustrative "ascii_upper" and "ascii_length" scalar string functions valid for Array and Scalar inputs #7278

wesm wants to merge 3 commits into from

Conversation

@wesm
Copy link
Member

@wesm wesm commented May 26, 2020 •
edited
Loading

There's some new code generation machinery here (that will be worth ongoing iteration) but the relevant implementation / "developer UX" is what's in string_scalar_ascii.cc, take a look.

Note: the implementation of ascii_upper is far from optimal. std::toupper does more than convert ASCII to uppercase and so it would likely be faster to replace it with a bespoke implementation that only deals with the ASCII alphabetic character space

In [1]: import pyarrow as pa; import pyarrow.compute as pc                                                                                                                                     

In [2]: arr = pa.array(['aaa', 'bbbbbb', None, ''])                                                                                                                                            

In [3]: pc.ascii_upper(arr)                                                                                                                                                                    
Out[3]: 
<pyarrow.lib.StringArray object at 0x7f7044003e50>
[
  "AAA",
  "BBBBBB",
  null,
  ""
]

In [4]: pc.ascii_length(arr)                                                                                                                                                                   
Out[4]: 
<pyarrow.lib.Int32Array object at 0x7f7044003910>
[
  3,
  6,
  null,
  0
]

int64 offsets are respected with LargeString

In [5]: arr = pa.array(['aaa', 'bbbbbb', None, ''], type='large_utf8')                                                                                                                         

In [6]: pc.ascii_length(arr)                                                                                                                                                                   
Out[6]: 
<pyarrow.lib.Int64Array object at 0x7f703c74cbb0>
[
  3,
  6,
  null,
  0
]

@wesm
Copy link
Member Author

wesm commented May 26, 2020

cc @maartenbreddels, have a look at arrow/compute/kernels/scalar_string_ascii.cc for the example function implementations (valid both for 32-bit and 64-bit offset string types). As more functions are added, common structures will emerge to enable implementing and testing them to be easier and easier.

@wesm wesm changed the title ARROW-8922: [C++] Add illustrative "ascii_upper" and "ascii_length" scalar functions valid for Array and Scalar inputs ARROW-8922: [C++] Add illustrative "ascii_upper" and "ascii_length" scalar string functions valid for Array and Scalar inputs May 26, 2020
@github-actions
Copy link

github-actions bot commented May 26, 2020

https://issues.apache.org/jira/browse/ARROW-8922

@wesm
Copy link
Member Author

wesm commented May 31, 2020

+1. I'm doing some cleanup of includes (and handling moving ArrayData to a separate header) and it will be helpful to me to have this merged. Please leave comments and I will address them in follow up

@wesm wesm closed this in f0ed8e2 May 31, 2020
@wesm wesm deleted the ARROW-8922 branch May 31, 2020 12:56
@asfimport asfimport mentioned this pull request May 31, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wesm