Skip to content

[WIP] Continuous snapshot data loss#7295

Closed
agourlay wants to merge 1 commit intodevfrom
continuous-snapshot-data-loss
Closed

[WIP] Continuous snapshot data loss#7295
agourlay wants to merge 1 commit intodevfrom
continuous-snapshot-data-loss

Conversation

@agourlay
Copy link
Member

RUST_LOG=debug cargo nextest run --all continuous --nocapture

────────────
 Nextest run ID 8c335d13-b250-42b7-9da4-32c74ff69005 with nextest profile: default
    Starting 1 test across 22 binaries (1350 tests skipped)
       START             collection::integration continuous_snapshot_test::test_continuous_snapshot

running 1 test
[2025-09-23T14:10:41Z INFO  wal] Wal { path: "/tmp/test_collectionf8Rfmt/0/wal", segment-count: 1, entries: [0, 0)  }: opened
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  collection::collection::snapshots] Creating collection snapshot test-0-2025-09-23-14-10-41.snapshot into "/tmp/test_snapshotstnfDIe/test-0-2025-09-23-14-10-41.snapshot"
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z INFO  shard::proxy_segment::snapshot_entry] Taking a snapshot of a proxy segment
[2025-09-23T14:10:41Z WARN  collection::collection_manager::collection_updater] Update operation declined: No point with id 2 found
[2025-09-23T14:10:41Z WARN  collection::shards::replica_set::update] Failed to update shard test:0 on peer 0, error: No point with id 2 found

thread 'continuous_snapshot_test::test_continuous_snapshot' panicked at lib/collection/tests/integration/continuous_snapshot_test.rs:224:31:
points_task error: No point with id 2 found

@agourlay
Copy link
Member Author

For the record, this particular failing test has been fixed by #7298

Closing the PR.

@agourlay agourlay closed this Sep 25, 2025
@agourlay
Copy link
Member Author

working on a new different repro #7308

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant