Skip to content

GH-33976: [Python] Add scan ExecNode options#34530

Merged
jorisvandenbossche merged 1 commit intoapache:mainfrom
jorisvandenbossche:gh-33976-dataset
Mar 14, 2023
Merged

GH-33976: [Python] Add scan ExecNode options#34530
jorisvandenbossche merged 1 commit intoapache:mainfrom
jorisvandenbossche:gh-33976-dataset

Conversation

@jorisvandenbossche
Copy link
Copy Markdown
Member

@jorisvandenbossche jorisvandenbossche commented Mar 10, 2023

Continuing GH-34102, this adds the exec node options classes defined in the dataset module (scan, not yet write).

@github-actions
Copy link
Copy Markdown

Comment on lines +329 to +330
# "b" is still included, but without data as it will be removed by the projection
assert pc.all(result["b"].is_null()).as_py()
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@westonpace this was a bit surprising, but expected at the moment? And that is something that the ScanV2Options tackle?

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Mar 10, 2023
@jorisvandenbossche
Copy link
Copy Markdown
Member Author

Going to merge this so I can continue with #34401.

@jorisvandenbossche jorisvandenbossche merged commit 3a6fc1f into apache:main Mar 14, 2023
@jorisvandenbossche jorisvandenbossche deleted the gh-33976-dataset branch March 14, 2023 08:41
@ursabot
Copy link
Copy Markdown

ursabot commented Mar 14, 2023

Benchmark runs are scheduled for baseline = f1eece9 and contender = 3a6fc1f. 3a6fc1f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.48% ⬆️0.0%] test-mac-arm
[Finished ⬇️0.51% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.28% ⬆️0.03%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 3a6fc1f9 ec2-t3-xlarge-us-east-2
[Finished] 3a6fc1f9 test-mac-arm
[Finished] 3a6fc1f9 ursa-i9-9960x
[Finished] 3a6fc1f9 ursa-thinkcentre-m75q
[Finished] f1eece9f ec2-t3-xlarge-us-east-2
[Finished] f1eece9f test-mac-arm
[Finished] f1eece9f ursa-i9-9960x
[Finished] f1eece9f ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link
Copy Markdown

ursabot commented Mar 14, 2023

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants