Project

General

Profile

Actions

Bug #71434

open

Test failure: test_for_6_ongoing_clones (tasks.cephfs.volumes.test_clone_stats.TestOngoingClonesCounter)

Added by Venky Shankar 10 months ago. Updated 6 months ago.

Status:
New
Priority:
Urgent
Assignee:
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Q/A
Backport:
tentacle,squid
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
mgr/volumes
Labels (FS):
Pull request ID:
Tags (freeform):
backport_processed
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

https://pulpito.ceph.com/vshankar-2025-05-13_20:37:42-fs-wip-vshankar-testing-20250513.090120-tentacle-debug-testing-default-smithi/8283915/

The test expects string "6 ongoing clones" in the progress_events JSON, but that never happened (maybe some clones finished?).

8283915 grep "ongoing clones" teuthology.log
2025-05-13T23:47:35.796 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "1 ongoing clones - average progress is 30.859% (8s)\n      [========....................] (remaining: 18s)",
2025-05-13T23:49:57.463 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "1 ongoing clones - average progress is 4.766% (1s)\n      [=...........................] (remaining: 22s)",
2025-05-13T23:52:05.629 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "1 ongoing clones - average progress is 1.328% (0s)\n      [............................] ",
2025-05-13T23:54:25.808 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "1 ongoing clones - average progress is 4.766% (1s)\n      [=...........................] (remaining: 22s)",
2025-05-13T23:55:44.116 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 32.357% (7s)\n      [=========...................] (remaining: 15s)",
2025-05-13T23:55:46.020 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 32.357% (7s)\n      [=========...................] (remaining: 15s)",
2025-05-13T23:55:47.992 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 16.927% (10s)\n      [====........................] (remaining: 53s)",
2025-05-13T23:57:18.956 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "1 ongoing clones - average progress is 4.427% (0s)\n      [=...........................] ",
2025-05-13T23:57:20.939 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 33.854% (7s)\n      [=========...................] (remaining: 13s)",
2025-05-13T23:57:22.974 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 50.521% (9s)\n      [==============..............] (remaining: 9s)",
{'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 50.521% (9s)\n      [==============..............] (remaining: 9s)', 'progress': 0.5052099823951721, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 40.417% (0s)\n      [===========.................] ', 'progress': 0.4041700065135956, 'add_to_ceph_s': True}}
2025-05-13T23:57:59.307 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "3 ongoing clones - average progress is 33.333% (44s)\n      [=========...................] (remaining: 89s)",
{'mgr-vol-ongoing-clones': {'message': '3 ongoing clones - average progress is 33.333% (44s)\n      [=========...................] (remaining: 89s)', 'progress': 0.3333300054073334, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 8 clones - average progress is 26.107% (34s)\n      [=======.....................] (remaining: 98s)', 'progress': 0.26107001304626465, 'add_to_ceph_s': True}}
2025-05-14T00:05:49.538 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "1 ongoing clones - average progress is 4.427% (0s)\n      [=...........................] ",
2025-05-14T00:05:51.465 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "2 ongoing clones - average progress is 29.557% (3s)\n      [========....................] (remaining: 7s)",
2025-05-14T00:07:29.859 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 42.252% (8s)\n      [===========.................] (remaining: 11s)",
2025-05-14T00:07:31.723 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 42.252% (8s)\n      [===========.................] (remaining: 11s)",
2025-05-14T00:07:33.732 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 59.505% (14s)\n      [================............] (remaining: 10s)",
2025-05-14T00:09:50.266 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 84.961% (24s)\n      [=======================.....] (remaining: 4s)",
2025-05-14T00:09:52.120 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 84.961% (24s)\n      [=======================.....] (remaining: 4s)",
2025-05-14T00:09:53.920 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 84.961% (24s)\n      [=======================.....] (remaining: 4s)",
2025-05-14T00:09:55.890 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 84.961% (24s)\n      [=======================.....] (remaining: 4s)",
2025-05-14T00:09:57.704 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 83.789% (48s)\n      [=======================.....] (remaining: 9s)",
2025-05-14T00:09:59.557 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 83.789% (48s)\n      [=======================.....] (remaining: 9s)",
2025-05-14T00:10:01.400 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 83.789% (48s)\n      [=======================.....] (remaining: 9s)",
2025-05-14T00:10:03.336 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 83.789% (48s)\n      [=======================.....] (remaining: 9s)",
2025-05-14T00:10:05.192 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 83.789% (48s)\n      [=======================.....] (remaining: 9s)",
2025-05-14T00:10:07.129 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "4 ongoing clones - average progress is 83.789% (48s)\n      [=======================.....] (remaining: 9s)",
2025-05-14T00:10:08.977 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 40.365% (58s)\n      [===========.................] (remaining: 87s)",
2025-05-14T00:10:10.816 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 40.365% (58s)\n      [===========.................] (remaining: 87s)",
2025-05-14T00:10:12.668 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 40.365% (58s)\n      [===========.................] (remaining: 87s)",
2025-05-14T00:10:14.512 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 40.365% (58s)\n      [===========.................] (remaining: 87s)",
2025-05-14T00:10:16.334 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 56.25% (66s)\n      [===============.............] (remaining: 51s)",
2025-05-14T00:10:18.096 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 56.25% (66s)\n      [===============.............] (remaining: 51s)",
2025-05-14T00:10:19.903 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 56.25% (66s)\n      [===============.............] (remaining: 51s)",
2025-05-14T00:10:21.725 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "5 ongoing clones - average progress is 70.261% (72s)\n      [===================.........] (remaining: 30s)",
2025-05-14T00:10:23.611 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "3 ongoing clones - average progress is 62.847% (74s)\n      [=================...........] (remaining: 44s)",
2025-05-14T00:10:25.440 INFO:teuthology.orchestra.run.smithi137.stdout:            "message": "3 ongoing clones - average progress is 70.486% (76s)\n      [===================.........] (remaining: 31s)",

Related issues 1 (0 open1 closed)

Related to Ceph QA - QA Run #71305: wip-vshankar-testing-20250513.090120-tentacle-debugQA ClosedVenky ShankarActions
Actions #1

Updated by Venky Shankar 10 months ago

  • Related to QA Run #71305: wip-vshankar-testing-20250513.090120-tentacle-debug added
Actions #2

Updated by Venky Shankar 9 months ago

@Rishabh Dave Please do an RCA of this defect.

Actions #4

Updated by Rishabh Dave 8 months ago

  • Status changed from New to Fix Under Review
Actions #5

Updated by Venky Shankar 7 months ago

  • Status changed from Fix Under Review to Pending Backport
  • Backport changed from tentacle to tentacle,squid
Actions #6

Updated by Upkeep Bot 7 months ago

  • Copied to Backport #72605: squid: Test failure: test_for_6_ongoing_clones (tasks.cephfs.volumes.test_clone_stats.TestOngoingClonesCounter) added
Actions #7

Updated by Upkeep Bot 7 months ago

  • Copied to Backport #72606: tentacle: Test failure: test_for_6_ongoing_clones (tasks.cephfs.volumes.test_clone_stats.TestOngoingClonesCounter) added
Actions #8

Updated by Upkeep Bot 7 months ago

  • Tags (freeform) set to backport_processed
Actions #9

Updated by Jos Collin 7 months ago

  • Status changed from Pending Backport to New

This doesn't have the PR attached. Moving to New state.

Actions #10

Updated by Jos Collin 7 months ago

  • Copied to deleted (Backport #72605: squid: Test failure: test_for_6_ongoing_clones (tasks.cephfs.volumes.test_clone_stats.TestOngoingClonesCounter))
Actions #11

Updated by Jos Collin 7 months ago

  • Copied to deleted (Backport #72606: tentacle: Test failure: test_for_6_ongoing_clones (tasks.cephfs.volumes.test_clone_stats.TestOngoingClonesCounter))
Actions #12

Updated by Venky Shankar 6 months ago

  • Pull request ID set to 64731

Fixed missing pull request id.

Actions

Also available in: Atom PDF