Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Jan 26, 2025

Which issue does this PR close?

Rationale for this change

DataFusion is largely intended to be used via the datafusion crate. It is split into multiple subcrates largely for project organization rather than for users to use subcrates

The examples currently use many of the datafusion subcrates (e.g. datafusion_common) directly. This is non ideal because

  1. It may confuse users who try to follow the examples and lead them incorrectly towards using the subcrates directly
  2. It may potentially mask issues where a struct is not publicly exported

Thus I think our examples should only use the DataFusion crate

What changes are included in this PR?

  1. Update the examples to use (only) datafusion
  2. Re-export ApplyOrder in datafusion::optimizer to make it easier to use
  3. Re-export datafusion_physical_optimizer as datafusion::physical_optimizer for consistency

Are these changes tested?

Yes, by CI

Are there any user-facing changes?

  1. Better examples
  2. Users can use datafusion::optimizer::ApplyOrder in addition to what they currently do use datafusion::optimizer::optimizer::ApplyOrder

@github-actions github-actions bot added the optimizer Optimizer rules label Jan 26, 2025
@alamb alamb marked this pull request as ready for review January 26, 2025 13:23
@github-actions github-actions bot added the core Core DataFusion crate label Jan 26, 2025
@alamb alamb changed the title Only use datafusion crate in the examples Export datafusion_physical_optimizer, only use datafusion crate in the examples Jan 26, 2025
Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @alamb. The description and changes make sense to me

@berkaysynnada berkaysynnada merged commit 0eebc0c into apache:main Jan 27, 2025
29 checks passed
@alamb alamb deleted the alamb/only_datafusion_in_examples branch January 27, 2025 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate optimizer Optimizer rules

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants