Skip to content

Ends ParquetRecordBatchStream when polling on StreamState::Error#3404

Merged
viirya merged 2 commits into
apache:masterfrom
viirya:remove_unnecessary_state
Dec 28, 2022
Merged

Ends ParquetRecordBatchStream when polling on StreamState::Error#3404
viirya merged 2 commits into
apache:masterfrom
viirya:remove_unnecessary_state

Conversation

@viirya

@viirya viirya commented Dec 28, 2022

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #.

Rationale for this change

While I'm working on other stuff which also uses Stream, I found on this StreamState::Error returning Poll::Pending without scheduling a wake for the task will cause the stream execution stuck if continuous poll is called after error happens.

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions Bot added the parquet Changes to the parquet crate label Dec 28, 2022
@tustvold

Copy link
Copy Markdown
Contributor

the result of poll already reflects the error and we won't go for next poll

This is only if using a method that inspects the return types. If you used something like collect instead of try_collect you would just continue going after the error

Actually returning Poll::Pending without scheduling a wake for the task will cause the stream execution stuck if such poll is called.

Perhaps we could return None here?

@viirya

viirya commented Dec 28, 2022

Copy link
Copy Markdown
Member Author

Perhaps we could return None here?

Sounds good.

@viirya viirya changed the title Remove unnecessary StreamState::Error Ends ParquetRecordBatchStream when polling on StreamState::Error Dec 28, 2022
@viirya viirya merged commit 513d543 into apache:master Dec 28, 2022
@ursabot

ursabot commented Dec 28, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 8f99692 and contender = 513d543. 513d543 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants