Skip to content

Datastore: don't eagerly sort in bucket split rountine on ingestion path#4417

Merged
teh-cmc merged 1 commit intomainfrom
cmc/split_early_sort
Dec 1, 2023
Merged

Datastore: don't eagerly sort in bucket split rountine on ingestion path#4417
teh-cmc merged 1 commit intomainfrom
cmc/split_early_sort

Conversation

@teh-cmc
Copy link
Copy Markdown
Contributor

@teh-cmc teh-cmc commented Dec 1, 2023 •
edited
Loading

This should fix #4415, although without knowing how exactly the user ended up in that situation it's hard to say for certain.

Checklist

  • I have read and agree to Contributor Guide and the Code of Conduct
  • I've included a screenshot or gif (if applicable)
  • I have tested the web demo (if applicable):
    • Full build: app.rerun.io
    • Partial build: app.rerun.io - Useful for quick testing when changes do not affect examples in any way
  • The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG

@teh-cmc teh-cmc added 🪳 bug Something isn't working ⛃ re_datastore affects the datastore itself 📉 performance Optimization, memory use, etc include in changelog labels Dec 1, 2023
@teh-cmc teh-cmc merged commit ec77988 into main Dec 1, 2023
@teh-cmc teh-cmc deleted the cmc/split_early_sort branch December 1, 2023 12:21
@emilk emilk mentioned this pull request Dec 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@emilk emilk emilk approved these changes

Assignees

No one assigned

Labels

🪳 bug Something isn't working include in changelog 📉 performance Optimization, memory use, etc ⛃ re_datastore affects the datastore itself

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Logging lots of out-of-order data using the same timepoint on a timeline leads to extreme slowdown in datastore

2 participants

@teh-cmc @emilk