Skip to content

TBS: Add sampling.tail.discard_on_write_failure config#15159

Merged
carsonip merged 9 commits intoelastic:mainfrom
carsonip:index-on-failure-config
Jan 7, 2025
Merged

TBS: Add sampling.tail.discard_on_write_failure config#15159
carsonip merged 9 commits intoelastic:mainfrom
carsonip:index-on-failure-config

Conversation

@carsonip
Copy link
Member

@carsonip carsonip commented Jan 7, 2025

Motivation/summary

Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

Checklist

- [ ] Update CHANGELOG.asciidoc backported
- [ ] Documentation has been updated intentionally undocumented

For functional changes, consider:

  • Is it observable through the addition of either logging or metrics?
  • Is its use being published in telemetry to enable product improvement?
  • Have system tests been added to avoid regression?

How to test these changes

Enable TBS, trigger storage limit reached, check if behavior aligns with config = {true, false}

Related issues

Fixes #11127

@carsonip carsonip requested a review from a team as a code owner January 7, 2025 17:42
@mergify
Copy link
Contributor

mergify bot commented Jan 7, 2025

This pull request does not have a backport label. Could you fix it @carsonip? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-7.17 is the label to automatically backport to the 7.17 branch.
  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
  • backport-8.x is the label to automatically backport to the 8.x branch.

@mergify
Copy link
Contributor

mergify bot commented Jan 7, 2025

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label.

@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Jan 7, 2025
@carsonip carsonip requested a review from lahsivjar January 7, 2025 17:43
@carsonip carsonip added backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify labels Jan 7, 2025
lahsivjar
lahsivjar previously approved these changes Jan 7, 2025
Copy link
Contributor

@lahsivjar lahsivjar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@carsonip carsonip enabled auto-merge (squash) January 7, 2025 21:15
1pkg
1pkg previously approved these changes Jan 7, 2025
Copy link
Contributor

@1pkg 1pkg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be really nice to have a test for DiscardOnWriteFailure case.

@carsonip carsonip disabled auto-merge January 7, 2025 21:20
@carsonip carsonip dismissed stale reviews from 1pkg and lahsivjar via 594692c January 7, 2025 21:46
@carsonip carsonip requested a review from 1pkg January 7, 2025 21:47
@carsonip carsonip merged commit 9b40574 into elastic:main Jan 7, 2025
mergify bot pushed a commit that referenced this pull request Jan 7, 2025
Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

(cherry picked from commit 9b40574)
mergify bot pushed a commit that referenced this pull request Jan 7, 2025
Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

(cherry picked from commit 9b40574)
mergify bot pushed a commit that referenced this pull request Jan 7, 2025
Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

(cherry picked from commit 9b40574)
mergify bot added a commit that referenced this pull request Jan 7, 2025
…5172)

Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

(cherry picked from commit 9b40574)

Co-authored-by: Carson Ip <carsonip@users.noreply.github.com>
mergify bot added a commit that referenced this pull request Jan 7, 2025
…5170)

Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

(cherry picked from commit 9b40574)

Co-authored-by: Carson Ip <carsonip@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
mergify bot added a commit that referenced this pull request Jan 7, 2025
…5171)

Add config sampling.tail.discard_on_write_failure (default=false) for users to opt in to data loss when TBS storage limit is reached, as an escape hatch when increased ES indexing load is unacceptable.

(cherry picked from commit 9b40574)

Co-authored-by: Carson Ip <carsonip@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@endorama endorama mentioned this pull request Feb 4, 2025
1 task
@1pkg 1pkg self-assigned this Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-8.x Automated backport to the 8.x branch with mergify backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify test-plan test-plan-ok v8.18.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configurable option to handle events failed to be processed by TBS

4 participants