Is your feature request related to a problem?

One of the goals of extensibility is the ability to add/update/remove an extension without restarting the OpenSearch node.

From #2447:

What would the customer like to see/use:
c. Install/Update/Remove an extension without restarting OpenSearch.

Present design does not permit this:

• The extension discovery mechanism requires extensions to already be up and running before OpenSearch is started
• There is no mechanism to retry a failed connection, or to periodically poll an extension to ensure it is responsive
• Many of the modules associated with plugins assume a one-time initialization at startup with final, immutable data structures
• Other modules and data structures permit additive registration (e.g, the RestController, Settings, XContent) but do not permit "unregistering"
  • This problem will only get bigger when we add more extension points

What solution would you like?

• [FEATURE] Create a system index for tracking extensions #357
  • store data corresponding to the extensions.yml file
  • allow option for this index to be priority for persistent extensions list rather than config file
  • Use this index rather than the existing maps/lists in ExtensionsManager
• [FEATURE] Create REST API calls to add, update, or remove an extension #358
  • Update the index
  • Optionally allow user to download updated extensions.yml file
  • Stop the extension when removed; reboot extension when updated
    • Start up an extension on local node if so configured
    • Give user instructions on starting an extension on a remote node
• [FEATURE] Establish periodic polling of extensions to verify connectivity #359
  • Allow user to configure auto-restart
• [FEATURE] Update existing extension point registration to permit both adding and removing #360
  • Do this as part of future extension point implementation

What alternatives have you considered?

Creating an external process that manages the extensions (an extension managing extensions!)

Do you have any additional context?

Previously mentioned in #174