Skip to content

Extend profile capabilities to plugins#18656

Merged
reta merged 15 commits intoopensearch-project:mainfrom
neuenfeldttj:profile-plugins
Jul 17, 2025
Merged

Extend profile capabilities to plugins#18656
reta merged 15 commits intoopensearch-project:mainfrom
neuenfeldttj:profile-plugins

Conversation

@neuenfeldttj
Copy link
Copy Markdown
Contributor

Description

This continues extending the profiling with plugins. Now that we have ProfileMetrics, we can have plugins piggy-back off of the default query profiler and provide a hashmap of metric suppliers. Plugins that provide queries can add profile metrics and profile what they want. They achieve this by getting the top breakdown context (since the default query handles the profile tree creation). Because of the new plugin extension, there will be API changes!

Related Issues

Resolves #18460

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added discuss Issues intended to help drive brainstorming and decision making enhancement Enhancement or improvement to existing feature or request Search:Query Insights labels Jul 1, 2025
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 1, 2025

✅ Gradle check result for cef03ba: SUCCESS

@codecov
Copy link
Copy Markdown

codecov bot commented Jul 1, 2025

Codecov Report

Attention: Patch coverage is 76.74419% with 10 lines in your changes missing coverage. Please review.

Project coverage is 72.82%. Comparing base (fc6b08e) to head (f7c4bd0).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...main/java/org/opensearch/search/SearchService.java 16.66% 5 Missing ⚠️
.../java/org/opensearch/search/profile/Profilers.java 66.66% 2 Missing ⚠️
...ensearch/search/internal/ContextIndexSearcher.java 0.00% 1 Missing ⚠️
...ch/search/profile/AbstractInternalProfileTree.java 75.00% 0 Missing and 1 partial ⚠️
.../search/profile/query/ConcurrentQueryProfiler.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #18656      +/-   ##
============================================
- Coverage     72.86%   72.82%   -0.05%     
+ Complexity    68571    68541      -30     
============================================
  Files          5566     5566              
  Lines        314513   314701     +188     
  Branches      45636    45653      +17     
============================================
+ Hits         229167   229170       +3     
- Misses        66789    66999     +210     
+ Partials      18557    18532      -25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 2, 2025

❌ Gradle check result for d723702: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 2, 2025

✅ Gradle check result for 3d13284: SUCCESS

@reta
Copy link
Copy Markdown
Contributor

reta commented Jul 15, 2025

@neuenfeldttj thank you for pulling it off, a super minor comment but LGTM otherwise, we need to have build green, could you please work on that? @andrross @msfroh if you have some spare cycles to look at it, would be great, thank you.

neuenfeldttj and others added 2 commits July 15, 2025 19:27
Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
Signed-off-by: TJ Neuenfeldt <56700214+neuenfeldttj@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 10db8a4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: TJ Neuenfeldt <56700214+neuenfeldttj@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 21d1030: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

✅ Gradle check result for d16e061: SUCCESS

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 73ac38b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for aa4f55d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
Copy link
Copy Markdown
Member

@andrross andrross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one nitpick, otherwise looks good

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for f7c4bd0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

✅ Gradle check result for f7c4bd0: SUCCESS

Copy link
Copy Markdown
Contributor

@reta reta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @neuenfeldttj !

@reta reta merged commit cd4388a into opensearch-project:main Jul 17, 2025
32 of 34 checks passed
@reta reta added the v3.2.0 label Jul 17, 2025
pranikum pushed a commit to pranikum/OpenSearch that referenced this pull request Jul 17, 2025
* port over plugin extension, double check with knn tests

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* changelog, added tests, changed to contexualpb for context

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* spotless

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* renamed, made extension point a function

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* changelog, default ctors

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* added getter, changed to findProfileBreakdown

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* changelog and abstractpb

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* trigger

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* revert breaking change

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* trigger

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* trigger build

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* search module nit fix

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* spotless

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

---------

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
Signed-off-by: TJ Neuenfeldt <56700214+neuenfeldttj@users.noreply.github.com>
tandonks pushed a commit to tandonks/OpenSearch that referenced this pull request Aug 5, 2025
* port over plugin extension, double check with knn tests

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* changelog, added tests, changed to contexualpb for context

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* spotless

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* renamed, made extension point a function

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* changelog, default ctors

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* added getter, changed to findProfileBreakdown

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* changelog and abstractpb

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* trigger

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* revert breaking change

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* trigger

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* trigger build

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* search module nit fix

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

* spotless

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

---------

Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>
Signed-off-by: TJ Neuenfeldt <56700214+neuenfeldttj@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

discuss Issues intended to help drive brainstorming and decision making enhancement Enhancement or improvement to existing feature or request Search:Query Insights v3.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC] Profiling Extensibility

4 participants