Ensure DLM only runs one general loop at a time#143883
Conversation
Since DLM only runs at an interval, we'd like a second triggering of the schedule not to run DLM again, so that we can limit the overhead of DLM an avoid concurrent running issues. This commit adds a "currently running" flag that protects DLM from being run concurrently on short schedules.
|
Hi @dakrone, I've created a changelog YAML for you. |
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
seanzatzdev
left a comment
There was a problem hiding this comment.
LGTM, thanks for working on this
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis pull request adds concurrency protection to the Data Stream Lifecycle Manager (DLM) service. The DataStreamLifecycleService.run method now uses an AtomicBoolean flag to ensure only one execution runs simultaneously. If a run is already in progress, the invocation exits with a debug log. Exception handling is enhanced per-project to prevent single failures from terminating the entire run. A new test validates this single-run-at-a-time behavior using latches to coordinate concurrent thread execution. A changelog entry documents this enhancement. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Since DLM only runs at an interval, we'd like a second triggering of the schedule not to run DLM again, so that we can limit the overhead of DLM an avoid concurrent running issues. This commit adds a "currently running" flag that protects DLM from being run concurrently on short schedules.