-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Is your feature request related to a problem or challenge?
Part of #12550
While working on https://github.com/datafusion-contrib/datafusion-dft I want to register various types of extensions while configuring the SessionContext, ideally adding each extension to the SessionStateBuilder each time.
However, I found that there were a few APIs missing on SessionStateBuilder so I had to implement my own workaround builder here: https://github.com/datafusion-contrib/datafusion-dft/blob/8247555f9464058c1ac3370196739ac2b19343ee/src/extensions/builder.rs#L102-L1078
And then call it
https://github.com/datafusion-contrib/datafusion-dft/blob/8247555f9464058c1ac3370196739ac2b19343ee/src/extensions/s3.rs#L59-L62
SessionStateBuilder has no way to register an object store.
Describe the solution you'd like
I would like a way to register object stores
Also it should have
- Documentation
- Tests (ideally a doc test with an example of how to use to use it)
Describe alternatives you've considered
I recommend adding two new functions
SessionStateBuilder::with_object_storethat calls through to RuntimeEnv::register_object_store- SessionStateBuilder::runtime_env() that returns the current
RuntimeEnv(follow model here) which would permit access to the underlying RuntimeEnv for access to other more advanced features
So this would be used like
let state = SessionStateBuilder::new()
.with_object_store(url, object_store)
.build()Or
let mut builder = SessionStateBuilder::new();
bulder.runtime_env().register_object_store(url, object_store)
let state = builder.build()Additional context
No response