Skip to content

[system testing][logstash] Fix duplicate event entries when running with logstash#1497

Merged
bhapas merged 1 commit intoelastic:mainfrom
bhapas:fix-duplicate-docs
Oct 9, 2023
Merged

[system testing][logstash] Fix duplicate event entries when running with logstash#1497
bhapas merged 1 commit intoelastic:mainfrom
bhapas:fix-duplicate-docs

Conversation

@bhapas
Copy link
Copy Markdown
Contributor

@bhapas bhapas commented Oct 9, 2023

When the stack is pulled up with logstash having elastic_integration plugin , it does not pass on the _id of the document created when running ingest pipelines , leading to duplicate entries in elasticsearch

This PR solves the issue by setting the document_id in elasticsearch output.

How to test this duplication

  • Follow the steps mentioned here to setup the stack.
  • Navigate to test/packages/with-logstash/ti_misp
  • Run elastic-package test system -v [ Without this PR , the test fails as there is a deliberate duplicate event added to the system tests ]

@bhapas bhapas self-assigned this Oct 9, 2023
@bhapas bhapas requested review from a team, jsoriano and mrodm October 9, 2023 11:08
@elasticmachine
Copy link
Copy Markdown
Collaborator

💚 Build Succeeded

cc @bhapas

@bhapas bhapas merged commit 91fe607 into elastic:main Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants