Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Since object_store is "low" in the dependency stack, if we make a breaking change to the API, we then need to wait for that breaking change releases in all other downstream dependencies
So for example, it can take up to 4 months to get into DataFusion (wait 3 months for next major arrow release and then a month for the next major DataFusion release)
That timeframe means that any changes that require non breaking API changes take substantial time to arrive downstream.
Thus allowing new features to be added without breaking downstream changes is more valuable
For example, because PutOpts is a pub struct with pub fields, we can not add any new fields without a breaking API release
|
#[derive(Debug, Clone, Default)] |
|
pub struct PutOptions { |
|
/// Configure the [`PutMode`] for this operation |
|
pub mode: PutMode, |
|
/// Provide a [`TagSet`] for this object |
|
/// |
|
/// Implementations that don't support object tagging should ignore this |
|
pub tags: TagSet, |
|
/// Provide a set of [`Attributes`] |
|
/// |
|
/// Implementations that don't support an attribute should return an error |
|
pub attributes: Attributes, |
|
/// Implementation-specific extensions. Intended for use by [`ObjectStore`] implementations |
|
/// that need to pass context-specific information (like tracing spans) via trait methods. |
|
/// |
|
/// These extensions are ignored entirely by backends offered through this crate. |
|
/// |
|
/// They are also eclused from [`PartialEq`] and [`Eq`]. |
|
pub extensions: Extensions, |
|
} |
@crepererum added a CopyOptions to reduce replication in the API and to set us up for
However, we realized during review that adding cross bucked copy support would likely require adding a new fields to CopyOptions and thus another breaking API change.
Describe the solution you'd like
Some way that we can add to the existing options without breaking API changes
Describe alternatives you've considered
@tustvold suggests: #548 (comment) following the Attributes model:
|
pub enum Attribute { |
|
/// Specifies how the object should be handled by a browser |
|
/// |
|
/// See [Content-Disposition](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition) |
|
ContentDisposition, |
|
/// Specifies the encodings applied to the object |
|
/// |
|
/// See [Content-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding) |
|
ContentEncoding, |
|
/// Specifies the language of the object |
|
/// |
|
/// See [Content-Language](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Language) |
|
ContentLanguage, |
|
/// Specifies the MIME type of the object |
|
/// |
|
/// This takes precedence over any [ClientOptions](crate::ClientOptions) configuration |
|
/// |
|
/// See [Content-Type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type) |
|
ContentType, |
|
/// Overrides cache control policy of the object |
|
/// |
|
/// See [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control) |
|
CacheControl, |
|
/// Specifies the storage class of the object. |
|
/// |
|
/// See [AWS](https://aws.amazon.com/s3/storage-classes/), |
|
/// [GCP](https://cloud.google.com/storage/docs/storage-classes), and |
|
/// [Azure](https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-tier). |
|
/// `StorageClass` is used as the name for this attribute because 2 of the 3 storage providers |
|
/// use that name |
|
StorageClass, |
|
/// Specifies a user-defined metadata field for the object |
|
/// |
|
/// The String is a user-defined key |
|
Metadata(Cow<'static, str>), |
|
} |
I think the only way to achieve this would be to make the options enumerable, i.e. a list of some non_exhaustive enum, that way implementations can error if they encounter an option they don't recognise.
Effectively this would take the pattern of Attributes and apply it more broadly.
We could do this, but I think that's probably a separate ticket, and we'd want to do it globally.
Additional context
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Since object_store is "low" in the dependency stack, if we make a breaking change to the API, we then need to wait for that breaking change releases in all other downstream dependencies
So for example, it can take up to 4 months to get into DataFusion (wait 3 months for next major arrow release and then a month for the next major DataFusion release)
That timeframe means that any changes that require non breaking API changes take substantial time to arrive downstream.
Thus allowing new features to be added without breaking downstream changes is more valuable
For example, because
PutOptsis apubstruct withpubfields, we can not add any new fields without a breaking API releasearrow-rs-object-store/src/lib.rs
Lines 1619 to 1638 in 0661843
copy©_if_not_exists=>copy_opts#548 (comment)@crepererum added a
CopyOptionsto reduce replication in the API and to set us up forHowever, we realized during review that adding cross bucked copy support would likely require adding a new fields to
CopyOptionsand thus another breaking API change.Describe the solution you'd like
Some way that we can add to the existing options without breaking API changes
Describe alternatives you've considered
@tustvold suggests: #548 (comment) following the
Attributesmodel:arrow-rs-object-store/src/attributes.rs
Lines 25 to 60 in 0661843
Additional context