Skip to content

kvserver: sep-raft-log: add crash-testing datadriven test harness #93247

@tbg

Description

@tbg

We need to exhaustively test the invariants for the engine states in separate raft logs.
We'll add assertions but should also explore these using datadriven tests.

Some of the code in #88606 might be helpful.

TODO:

  • crash in the middle of snapshot ingestion
  • crash in the middle of replica subsumption (snapshot and merge trigger case)
  • crash in the middle of replica creation
  • crash in the middle of split
  • crash in the middle of merge
  • crash in the middle of replica destruction
  • general engine-level testing of all of the above.

I'll add that our testing of the snapshot SSTs is currently weak. We rely on spinning up a real cluster and then intercept the SSTs, which is not very maintainable. We should complement this with kvstorage-level tests that create a snapshot and examine in detail their contents and the engine each one targets.

Epic CRDB-55218

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-kvKV Team

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions