Updated to a meta issue

See this comment: #154 (comment)

Is your feature request related to a problem?

Part of #148.

The existing WriteableSettings implementation contains the minimum required for the AD Create Detector functionality. However, while it conveys the existing settings and default value, it loses functionality:

• The new default value is pre-calculated after evaluating existing fallbacks.
  • It does not have access to any settings, and is evaluated based on an empty settings object so could be incorrect if it depends on an environment setting
  • If the value of an environment setting or fallback setting changes later, the setting will be incorrect.
• It loses track of the parsing function. Although for the enumerated types this is standard, there is a functionality for users to create custom parsers, such has handling delimited strings, JSON, etc.
• It loses track of the validating function. While the initial setting for an integer will respect the provided min and max values, an update to the value will not be validated.

Internally, the default, parser, and validator are all functional interfaces. These can only be passed over transport using Java's internal serialization (implements Serializable).

Altough Setting implements toXContent it makes the same assumptions the WriteableSetting class did, pre-evaluating the default and ignoring parser and validator.

Complicating implementation (and the reason this task was deferred), the functional interfaces are not easily retrieved. While the default value is protected and could be retrieved by a subclass or same-package class, the parser and validator are private and would only be accessible by an inner class of the Setting class.

What solution would you like?

Known parts of the problem:

• The defaultValue, parser, and validator fields must be implemented in a serializable functional interface
  • ExtensionFunction extending Function or ExtensionValidator extending Validator that also extends/implements Serializable.
• Functionality to transfer the serialized bytes with the writeable and deserialize them appropriately when instantiating a new Setting in OpenSearch

Unknown: How to obtain access to the functional interfaces. Options include:

• Abusing reflection
  • I was okay doing this for a unit test but this should be a big no-no in production code
• For the defaultValue, create a subclass.
  • Since this doesn't help with the parser and validator, the subclass approach shouldn't be chosen just for this
• Change the access for parser and validator to package private
  • Probably the easiest option!
  • This would allow access from WriteableSetting in the same package, as it already has (but doesn't use) for defaultValue
• For all 3 functional interfaces, create a subclass as an inner class.
  • See AffixSetting for example, note that it uses delegate.defaultValue, and delegate.parser.
  • It may be possible to replace the entire WriteableSetting functionality internally, or at least the user-facing methods (leaving much of the existing implementation in a helper class)
  • A subclass of Setting without any special access could be created to handle use cases where the "pre-calculated default" and lack of validation require it
    • Could internally store the parser and validator for use by WriteableSetting
    • This could have the added value of also saving the generic type internally for later retrieval
    • Down side: mix of types
      • As a subclass, could exclusively use this type and delegate the default (unvalidated, standard parsing) calls to the superclass
    • Up side: can use same WriteableSetting implementation but the special treatment could use internal logic to handle the subclass differently

What alternatives have you considered?

Most are described above!

One other option is creating settings in OpenSearch via some REST API.

Do you have any additional context?

I would really prefer to avoid serialization but don't see any other option for functional interfaces.

There is a serializable functional interface in Apache Beam but I haven't dug into its implementation. I think it's much of what was described above, however, with limited API surface so isn't plug-and-play.