-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Is your feature request related to a problem or challenge?
As we start thinking of adding new catalog support in DataFusion for example, @goldmedal tried to do this for DynamicFileCatalog in #11035 , the current code structure requires that any such implementation be in datafusion/core which is already quite large and monolithic
Also, I think long term it would be a more sustainable structure if we can move ListingTable out of the core as well
However, this move is not likely possible until we move table provider and related catalog traits out of the core.
Also, if we could split up datafusion core more, it would be easier for people to use DataFusion as a smaller embedded library (aka not bring in file format support if they didn't need it)
Describe the solution you'd like
Ideally I would like the following traits / code moved into a datafusion-catalog crate:
TableProviderCatalogProviderSchemaProviderMemoryCatalogProviderInformationSchemaProvider
Describe alternatives you've considered
No response
Additional context
No response
Part of #10782