[WIP] Remove repositories.s3 settings and deprecated settings#23276
[WIP] Remove repositories.s3 settings and deprecated settings#23276dadoonet wants to merge 2 commits intoelastic:masterfrom
Conversation
We just skip the original test as it requires to have first registered a key and a secret in the vault.
|
I had planned to do this once a number of other things are fixed, including the refactoring necessary to load named configs eagerly at construction. Can we hold off on this? |
|
@rjernst Did you start the refactoring yet? If not I think that merging first that kind of PR can simplify your refactoring work. |
|
Yes, I have restarted the refactoring. I also have this PR sitting in a local branch. The reason I held off on this before the refactoring is I need to make it work with 5.x, so waiting to remove this allows for easy backporting, since at the moment 5.x and master are essentially the same. |
|
Fair enough. I doubt then that my PR is providing any help. |
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
|
@rjernst I'm closing this one as I doubt it will be picked up. In case it helps, you can pick that change https://github.com/elastic/elasticsearch/pull/23276/files#diff-c710699b84701013d83a5fe7ddc9a6d3 in your own branch so you won't have to rewrite the plugin changes documentation part. |
This PR is labeled as work in progress. Do not merge it as is.
Global repositories settings we were able to set in elasticsearch config file under
repositories.s3name space are removed (deprecate in 5.x). This includes
repositories.s3.bucket,repositories.s3.server_side_encryption,repositories.s3.buffer_size,repositories.s3.max_retries,repositories.s3.use_throttle_retries,repositories.s3.chunk_size,repositories.s3.compress,repositories.s3.storage_class,repositories.s3.canned_acl,repositories.s3.base_pathandrepositories.s3.path_style_access.We must set those settings per repository instead. Respectively
bucket,server_side_encryption,buffer_size,max_retries,use_throttle_retries,chunk_size,compress,storage_class,canned_acl,base_pathandpath_style_access.Related to #22800.
S3 plugin is now using named configurations. We basically define all the S3 clients we want to use by naming
them under
s3.client.xxxprefix wherexxxis the named configuration (usedefaultas the name if you want to haveone applied by default).
As a consequence, the following settings have been removed:
cloud.aws.access_key,cloud.aws.secret_key,cloud.aws.protocol,cloud.aws.proxy.host,cloud.aws.proxy.port,cloud.aws.proxy.username,cloud.aws.proxy.password,cloud.aws.signer,cloud.aws.read_timeout.Also their S3 specific equivalent have been removed:
cloud.aws.s3.access_key,cloud.aws.s3.secret_key,cloud.aws.s3.protocol,cloud.aws.s3.proxy.host,cloud.aws.s3.proxy.port,cloud.aws.s3.proxy.username,cloud.aws.s3.proxy.password,cloud.aws.s3.signer,cloud.aws.s3.read_timeout.Also
repositories.s3.access_key,repositories.s3.secret_key,repositories.s3.endpointandrepositories.s3.protocol.This also removes Signer as discussed in #22599. It will be marked as deprecated in 5.x branch.
Still to do:
s3.client.default.access_keyands3.client.default.secret_keyin the vault or to simulate it.