Skip to content

Introduce declarative plugin management#77544

Merged
pugnascotia merged 117 commits intoelastic:masterfrom
pugnascotia:70219-declarative-plugins
Nov 15, 2021
Merged

Introduce declarative plugin management#77544
pugnascotia merged 117 commits intoelastic:masterfrom
pugnascotia:70219-declarative-plugins

Conversation

@pugnascotia
Copy link
Copy Markdown
Contributor

@pugnascotia pugnascotia commented Sep 10, 2021

Closes #70219.

Introduce a declarative way for the Elasticsearch server to manage plugins, which reads the elasticsearch-plugins.yml file and works which plugins need to be added and / or removed to match the configuration. Also make it possible to configure a proxy in the config file, instead of through the environment.

Most of the work of adding and removing is still done in the InstallPluginAction and RemovePluginAction classes, so the behaviour should be the same as with the install and remove commands. However, these commands will now abort if the above config file exists. The intent is to make it harder for the configuration to drift.

This new method only applies to docker distribution types at the moment.

Since this syncing mechanism declarative, rather than imperative, the Cloud-specific plugin wrapper script is no longer required. Instead, an environment variable informs InstallPluginAction to install plugins from an archive directory instead of downloading them, where possible.

Also fix how the cloud-ess image depends on the cloud image.
@pugnascotia pugnascotia added >feature :Core/Infra/Plugins Plugin API and infrastructure v8.0.0 :Delivery/Cloud Cloud-specific packaging and deployment labels Sep 10, 2021
@pugnascotia
Copy link
Copy Markdown
Contributor Author

@mark-vieira I've added support for testing plugin installation on all Docker flavours, using the analysis-icu plugin. Can you check what I did with the task / configuration / artifact wiring, please?

I also tried to re-enable PluginCliTests, which I believe you last disabled, but something isn't working and I disabled them again.

Copy link
Copy Markdown
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Yeah, I need to go back and fix the plugin tests as a result of the example plugin refactoring.

@pugnascotia
Copy link
Copy Markdown
Contributor Author

I talked to Cloud Delivery and it doesn't look like the wrapper script is being used, so we're good to merge.

@pugnascotia
Copy link
Copy Markdown
Contributor Author

Oh, and I will add docs but I think that'll require some collaboration with the Docs folks, so I'll do that separately.

@pugnascotia pugnascotia merged commit 3018e52 into elastic:master Nov 15, 2021
@pugnascotia pugnascotia deleted the 70219-declarative-plugins branch November 15, 2021 14:41
pugnascotia added a commit to pugnascotia/elasticsearch that referenced this pull request Nov 15, 2021
Closes elastic#70219.

Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.

Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.

This new method only applies to `docker` distribution types at the
moment.

Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
elasticsearchmachine pushed a commit that referenced this pull request Nov 15, 2021
Closes #70219.

Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.

Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.

This new method only applies to `docker` distribution types at the
moment.

Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
@mark-vieira
Copy link
Copy Markdown
Contributor

I think we missed backporting this to 8.0 although we did to 7.16.

pugnascotia added a commit that referenced this pull request Nov 16, 2021
Closes #70219.

Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.

Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.

This new method only applies to `docker` distribution types at the
moment.

Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
@pugnascotia
Copy link
Copy Markdown
Contributor Author

Sorry - backported to 8.0 in 6d54e6f.

pugnascotia added a commit to pugnascotia/elasticsearch that referenced this pull request Nov 16, 2021
pugnascotia added a commit that referenced this pull request Nov 17, 2021
pugnascotia added a commit that referenced this pull request Nov 17, 2021
pugnascotia added a commit that referenced this pull request Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Plugins Plugin API and infrastructure :Delivery/Cloud Cloud-specific packaging and deployment >feature Team:Core/Infra Meta label for core/infra team Team:Delivery Meta label for Delivery team v8.0.0-rc1 v8.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide a declarative way to manage plugins

5 participants