This is a meta-issue to track the remaining work on the disk queue, which went beta in 7.10. This is for planning / prioritization as we push it towards GA, so anything that is a candidate for going into the first full release should be listed here.

This is in some ways a successor to #6859, although the new transactionless design and related simplifications means that much of that list is no longer applicable (transaction errors, page management, etc) or already complete in the current base package (full error handling / blocking), or is more advanced than we plan to include in v1 (multiple queues, advanced upgrade options and recovery tools...).

Phase 1

• Finalize on-disk schema ([libbeat] Track frame counts in saved segments in the disk queue #22970)
• Unit testing
  • Core loop logic ([libbeat] Fix potential deadlock in the disk queue + add more unit tests #21930, [libbeat] Add more disk queue unit tests and fix a size-check bug #22107)
  • Consumer ACKs
  • Helper goroutines (reader / writer / deleter)
  • Shutdown
• End-to-end tests

Phase 2

• Metrics
  • Report full queue metrics (including events from previous session) in monitoring.metrics.beat.libbeat.pipeline.events.active. #29296
  • Add additional queue-specific metrics?

Documentation with help of the doc team?

• Document the implementation logic (a lot is included in the comments and tests but there should be a standalone design summary) Describe what are the Guarantee are for the user.