Skip to content

Logstash Exporter implements custom publisher batch#46336

Merged
kaisecheng merged 12 commits intoelastic:mainfrom
kaisecheng:add_lsexp_batch
Sep 11, 2025
Merged

Logstash Exporter implements custom publisher batch#46336
kaisecheng merged 12 commits intoelastic:mainfrom
kaisecheng:add_lsexp_batch

Conversation

@kaisecheng
Copy link
Copy Markdown
Contributor

@kaisecheng kaisecheng commented Sep 1, 2025

Proposed commit message

Logstash exporter implements the publisher.Batch interface that wraps a plog.Logs, converting it to beats events and handling all the interface's functions it currently supports. With that done, we can fully reuse the beats/logstash clients, leveraging the existing clients backoff implementations, for example.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

kaisecheng and others added 2 commits August 30, 2025 00:10
@kaisecheng @edmocosta
implement batch and event
10150b5 
Co-authored-by: Edmo Vamerlatti Costa <edmo.vamerlatticosta@elastic.co>
@kaisecheng
add tests
875935a
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 1, 2025
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 1, 2025

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Sep 1, 2025

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @kaisecheng? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

@kaisecheng kaisecheng marked this pull request as ready for review September 1, 2025 17:16
@kaisecheng kaisecheng requested a review from a team as a code owner September 1, 2025 17:16
@kaisecheng kaisecheng added release-note:skip The PR should be ignored when processing the changelog Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Sep 1, 2025
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 1, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 1, 2025

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Sep 8, 2025

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add_lsexp_batch upstream/add_lsexp_batch
git merge upstream/main
git push upstream add_lsexp_batch

@khushijain21
Copy link
Copy Markdown
Contributor

khushijain21 commented Sep 10, 2025

also Adding @mauri870 as reviewer as Lee is on PTO

mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/batch.go
mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/batch_test.go Outdated
mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/batch_test.go Outdated
mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/event.go
mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/event_test.go Outdated
mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/event_test.go
mauri870
mauri870 reviewed Sep 10, 2025
View reviewed changes
Comment thread x-pack/otel/exporter/logstashexporter/internal/event.go
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Sep 11, 2025

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add_lsexp_batch upstream/add_lsexp_batch
git merge upstream/main
git push upstream add_lsexp_batch

@kaisecheng
Copy link
Copy Markdown
Contributor Author

kaisecheng commented Sep 11, 2025

@mauri870 Many thanks for the review and suggestion! This is ready for a second look.

@kaisecheng kaisecheng requested a review from mauri870 September 11, 2025 14:52
mauri870
mauri870 reviewed Sep 11, 2025
View reviewed changes
Comment thread x-pack/libbeat/outputs/otelconsumer/otelconsumer.go Outdated
mauri870
mauri870 approved these changes Sep 11, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@mauri870 mauri870 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks!

@mauri870 mauri870 requested a review from edmocosta September 11, 2025 18:27
@kaisecheng kaisecheng enabled auto-merge (squash) September 11, 2025 19:55
@kaisecheng kaisecheng merged commit afc53c0 into elastic:main Sep 11, 2025
207 checks passed
@khushijain21 khushijain21 added the backport-8.19 Automated backport to the 8.19 branch label Mar 9, 2026
mergify bot pushed a commit that referenced this pull request Mar 9, 2026
@kaisecheng @mergify
Logstash Exporter implements custom publisher batch (#46336)
b0193a2 
Logstash exporter implements the `publisher.Batch` interface that wraps a `plog.Logs`, converting it to beats events and handling all the interface's functions it currently supports. With that done, we can fully reuse the beats/logstash clients, leveraging the existing clients backoff implementations, for example.

---------

Co-authored-by: Edmo Vamerlatti Costa <edmo.vamerlatticosta@elastic.co>
(cherry picked from commit afc53c0)

# Conflicts:
#	x-pack/libbeat/outputs/otelconsumer/otelconsumer.go
#	x-pack/otel/exporter/logstashexporter/factory.go
@mergify mergify bot mentioned this pull request Mar 9, 2026
Closed
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mauri870 mauri870 mauri870 approved these changes

@leehinman leehinman Awaiting requested review from leehinman leehinman is a code owner automatically assigned from elastic/elastic-agent-data-plane

@khushijain21 khushijain21 Awaiting requested review from khushijain21 khushijain21 is a code owner automatically assigned from elastic/elastic-agent-data-plane

@edmocosta edmocosta Awaiting requested review from edmocosta

Assignees

@kaisecheng kaisecheng

Labels

backport-8.19 Automated backport to the 8.19 branch release-note:skip The PR should be ignored when processing the changelog Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@kaisecheng @elasticmachine @khushijain21 @mauri870 @edmocosta