Skip to content

Enabling experimental plugins specifically by plugin type and name#5676

Merged
dlvenable merged 2 commits intoopensearch-project:mainfrom
dlvenable:experimental-custom
May 7, 2025
Merged

Enabling experimental plugins specifically by plugin type and name#5676
dlvenable merged 2 commits intoopensearch-project:mainfrom
dlvenable:experimental-custom

Conversation

@dlvenable
Copy link
Copy Markdown
Member

@dlvenable dlvenable commented May 3, 2025

Description

Support enabling experimental plugins specifically by plugin type and name.

This also includes a change to the core plugin classes to allow them to define themselves as a plugin component type along with a name that is used to create the mapping in the YAML file.

Issues Resolved

Resolves #5675

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

… name. This also includes a change to the core plugin classes to allow them to define themselves as a plugin component type along with a name that is used to create the mapping in the YAML file.

Resolves opensearch-project#5675

Signed-off-by: David Venable <dlv@amazon.com>
*
* @return The name of the plugin component type.
*/
String value();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you have better to be an enum instead of a String?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I didn't use an enum is that this annotation is defined on the types themselves. For example, on the Source interface. I felt that making an enum would result in two places that need the modifications.

@san81
Copy link
Copy Markdown
Collaborator

san81 commented May 5, 2025

The mapping will go into a separate file? or additional entries in the data-prepper config file?

Copy link
Copy Markdown
Collaborator

@san81 san81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just asked two clarifying questions. Other than that, it looks good to me.

@dlvenable
Copy link
Copy Markdown
Member Author

The mapping will go into a separate file? or additional entries in the data-prepper config file?

This is part of data-prepper-config.yaml. It might look something like this:

ssl: false
serverPort: 4900
experimental:
  enabled:
    source:
      - rss
      - neptune
    processor:
      - ml_inference

san81
san81 previously approved these changes May 6, 2025
@JsonProperty("enable_all")
private boolean enableAll = false;

@JsonProperty("enabled")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name might be misunderstood as boolean flag. Maybe enabled_plugins?

Signed-off-by: David Venable <dlv@amazon.com>
@dlvenable dlvenable merged commit b5d7f37 into opensearch-project:main May 7, 2025
45 of 47 checks passed
@dlvenable dlvenable deleted the experimental-custom branch May 8, 2025 16:49
alparish pushed a commit to alparish/data-prepper that referenced this pull request May 22, 2025
…pensearch-project#5676)

Support enabling experimental plugins specifically by plugin type and name. This also includes a change to the core plugin classes to allow them to define themselves as a plugin component type along with a name that is used to create the mapping in the YAML file.

Resolves opensearch-project#5675

Signed-off-by: David Venable <dlv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support enabling specific experimental plugins

3 participants