Skip to content

[ObjC] Skip 0-length slices when writing to CFWriteStream (#37246)#37255

Closed
ziglerari wants to merge 1 commit intogrpc:masterfrom
ziglerari:cfstream-zero-length-buffers
Closed

[ObjC] Skip 0-length slices when writing to CFWriteStream (#37246)#37255
ziglerari wants to merge 1 commit intogrpc:masterfrom
ziglerari:cfstream-zero-length-buffers

Conversation

@ziglerari
Copy link
Copy Markdown
Contributor

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla bot commented Jul 19, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

for (size_t i = 0; i < data->Count(); i++) {
auto slice = data->RefSlice(i);
if (slice.size() == 0) {
continue;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sanity check is failing because of formatting, can you update accrodingly?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

continue;
}

size_t written_size =
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you mentioned in the issue, we should not use unsigned here but do error handling if the return value is -1.
I guess we can have a follow up PR to fix it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HannahShiSFB Could you recommend a fix here if it's very simple? If not we can always do a follow-up fix.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HannahShiSFB
Copy link
Copy Markdown
Collaborator

Thanks for the fix, BTW do you know in what cases that Arrow Flight needs to send a zero bytes buffer?

@ziglerari
Copy link
Copy Markdown
Contributor Author

Thanks for the fix, BTW do you know in what cases that Arrow Flight needs to send a zero bytes buffer?

@HannahShiSFB Arrow, in certain scenarios, utilizes a zero-byte buffer in conjunction with its Array structure. This is specifically related to the validity bitmap buffer, which is used to index valid entries (i.e., entries that are not null) within an Array. According to the Arrow specification, if all entries in an Array are valid, implementations may opt to indicate this by using a null buffer, effectively signaling that all entries are valid without explicitly storing this information. This approach is described as a "lazy" signaling mechanism for full validity (refer to Arrow Spec).
In its C++ implementation, Arrow takes advantage of this specification by inserting a null buffer into the body_buffers to denote that all entries in an Array are valid, as seen in the Arrow GitHub repository:
https://github.com/apache/arrow/blob/674e221f41c602c8f71c7a2c8e53e7c7c11b1ede/cpp/src/arrow/ipc/writer.cc#L167
These validity bitmaps, in turn, are serialized to Slices which are passed to grpc transport.

@ziglerari ziglerari force-pushed the cfstream-zero-length-buffers branch from 1d828d2 to 21c9db1 Compare July 24, 2024 22:09
Copy link
Copy Markdown
Contributor

@sampajano sampajano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change and detailed explanations! 😃

@ziglerari
Copy link
Copy Markdown
Contributor Author

@HannahShiSFB what prevents us from merging this PR? is there anything from my side?

@sampajano
Copy link
Copy Markdown
Contributor

@ziglerari Hi.. I think the PR is not being merged due to Python failure, which should be unrelated to this PR..

Let me re-trigger CI to see if it clears :)

@sampajano
Copy link
Copy Markdown
Contributor

UPDATE: Tests have cleared. Will merge soon! :)

copybara-service bot pushed a commit that referenced this pull request Aug 15, 2024
follow up of #37255

Closes #37344

COPYBARA_INTEGRATE_REVIEW=#37344 from HannahShiSFB:cfstream-write-error-handling c9a751d
PiperOrigin-RevId: 663329438
paulosjca pushed a commit to paulosjca/grpc that referenced this pull request Nov 25, 2024
grpc#37255)

grpc#37246

Closes grpc#37255

COPYBARA_INTEGRATE_REVIEW=grpc#37255 from ziglerari:cfstream-zero-length-buffers 21c9db1
PiperOrigin-RevId: 657770743
paulosjca pushed a commit to paulosjca/grpc that referenced this pull request Nov 25, 2024
follow up of grpc#37255

Closes grpc#37344

COPYBARA_INTEGRATE_REVIEW=grpc#37344 from HannahShiSFB:cfstream-write-error-handling c9a751d
PiperOrigin-RevId: 663329438
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants