Project

General

Profile

Actions

Bug #74332

open

ceph-kvstore-tool segfault in workunits/cephtool/test_kvstore_tool.sh

Added by Samuel Just 2 months ago. Updated 2 days ago.

Status:
Fix Under Review
Priority:
High
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Backport:
tentacle, squid, reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

2026-01-06T01:05:46.660 INFO:tasks.workunit.client.0.trial184.stderr:*** Caught signal (Segmentation fault) **
2026-01-06T01:05:46.660 INFO:tasks.workunit.client.0.trial184.stderr: in thread 7fde65e65a80 thread_name:ceph-kvstore-to
2026-01-06T01:05:46.664 INFO:tasks.workunit.client.0.trial184.stderr: ceph version 20.3.0-4698-g73712ccf (73712ccf0d7b99648a0c0d2ebf766adc13587a45) tentacle (dev - RelWithDebInfo)
2026-01-06T01:05:46.664 INFO:tasks.workunit.client.0.trial184.stderr: 1: /lib64/libc.so.6(+0x3fc30) [0x7fde6463fc30]
2026-01-06T01:05:46.664 INFO:tasks.workunit.client.0.trial184.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xc7) [0x7d8b57]
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, KeyValueDB::IteratorBounds)+0x2d5) [0x7d7935]
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 4: (BlueStore::_close_db()+0x44a) [0x62912a]
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 5: (BlueStore::umount()+0x1ac) [0x632a6c]
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 6: ceph-kvstore-tool() [0x5cdda3]
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 7: main()
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 8: /lib64/libc.so.6(+0x2a610) [0x7fde6462a610]
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 9: __libc_start_main()
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: 10: _start()
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr:2026-01-06T01:05:46.659+0000 7fde65e65a80 -1 *** Caught signal (Segmentation fault) **
2026-01-06T01:05:46.665 INFO:tasks.workunit.client.0.trial184.stderr: in thread 7fde65e65a80 thread_name:ceph-kvstore-to

https://qa-proxy.ceph.com/teuthology/yuriw-2026-01-05_23:00:35-rados-wip-yuri7-testing-2026-01-05-2021-distro-default-trial/1898/teuthology.log


Related issues 1 (1 open0 closed)

Related to Ceph - Bug #62079: ceph-kvstore-tool bluestore-kv destructive-repair at the s390xNew

Actions
Actions #1

Updated by Samuel Just 2 months ago

  • Related to Bug #62079: ceph-kvstore-tool bluestore-kv destructive-repair at the s390x added
Actions #2

Updated by Laura Flores about 2 months ago

  • Project changed from Ceph to RADOS
Actions #3

Updated by Laura Flores about 2 months ago

/a/lflores-2026-01-21_20:56:39-rados-main-distro-default-trial/11804

2026-01-22T00:36:40.650 INFO:tasks.workunit.client.0.trial014.stderr:2026-01-22T00:36:40.636+0000 7f5b24e14d40  1 bluestore(./cephtool.4Vc) _prepare_db_environment set db_paths to db,91804925952 db.slow,91804925952
2026-01-22T00:36:40.671 INFO:tasks.workunit.client.0.trial014.stdout:destructive-repair completed without reporting an error
2026-01-22T00:36:40.688 INFO:tasks.workunit.client.0.trial014.stderr:*** Caught signal (Segmentation fault) **
2026-01-22T00:36:40.688 INFO:tasks.workunit.client.0.trial014.stderr: in thread 7f5b24e14d40 thread_name:ceph-kvstore-to
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: ceph version 20.3.0-4860-g20763e3b (20763e3b00eca4f0dd8e8ba26cc85db51c13d48d) tentacle (dev - None)
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5b2601e520]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xb4) [0x557015530a24]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, KeyValueDB::IteratorBounds)+0x519) [0x557015531419]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 4: (BlueStore::_close_db()+0x31c) [0x5570153911ec]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 5: (BlueStore::umount()+0x230) [0x55701539d5d0]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 6: ceph-kvstore-tool(+0x202b13) [0x55701533fb13]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 7: main()
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 8: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b26005d90]
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 9: __libc_start_main()
2026-01-22T00:36:40.689 INFO:tasks.workunit.client.0.trial014.stderr: 10: _start()
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr:2026-01-22T00:36:40.660+0000 7f5b24e14d40 -1 *** Caught signal (Segmentation fault) **
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: in thread 7f5b24e14d40 thread_name:ceph-kvstore-to
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr:
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: ceph version 20.3.0-4860-g20763e3b (20763e3b00eca4f0dd8e8ba26cc85db51c13d48d) tentacle (dev - None)
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5b2601e520]
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xb4) [0x557015530a24]
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, KeyValueDB::IteratorBounds)+0x519) [0x557015531419]
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 4: (BlueStore::_close_db()+0x31c) [0x5570153911ec]
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 5: (BlueStore::umount()+0x230) [0x55701539d5d0]
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 6: ceph-kvstore-tool(+0x202b13) [0x55701533fb13]
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 7: main()
2026-01-22T00:36:40.690 INFO:tasks.workunit.client.0.trial014.stderr: 8: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b26005d90]
2026-01-22T00:36:40.691 INFO:tasks.workunit.client.0.trial014.stderr: 9: __libc_start_main()
2026-01-22T00:36:40.691 INFO:tasks.workunit.client.0.trial014.stderr: 10: _start()

Actions #4

Updated by Laura Flores about 2 months ago

  • Priority changed from Normal to High
Actions #5

Updated by Neha Ojha about 2 months ago

  • Assignee set to Jaya Prakash
Actions #6

Updated by Igor Fedotov about 2 months ago

  • Status changed from New to Fix Under Review
  • Assignee changed from Jaya Prakash to Igor Fedotov
  • Backport set to tentacle, squid, reef
  • Pull request ID set to 67069
Actions #7

Updated by Laura Flores about 2 months ago

/a/yuriw-2026-01-23_21:16:01-rados-wip-yuri13-testing-2026-01-23-1852-reef-distro-default-trial/15506

Actions #8

Updated by Laura Flores about 2 months ago

Scrub note: Under review

Actions #9

Updated by Laura Flores about 2 months ago

/a/yuriw-2026-01-23_21:16:01-rados-wip-yuri13-testing-2026-01-23-1852-reef-distro-default-trial/15445

Actions #10

Updated by Laura Flores about 2 months ago

/a/lflores-2026-01-23_19:07:45-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/15318

Actions #11

Updated by Aishwarya Mathuria about 2 months ago

/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13229
/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13347
/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13474
/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13518
/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13706
/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13896
/a/yuriw-2026-01-22_22:41:44-rados-wip-yuri10-testing-2026-01-22-2036-tentacle-distro-default-trial/13906

Actions #12

Updated by Laura Flores about 2 months ago

/a/lflores-2026-01-26_23:21:06-rados-wip-yuri12-testing-2026-01-22-2045-distro-default-trial/19106

+ others on the same run.

Actions #14

Updated by Laura Flores about 2 months ago

@Adam Kupczyk waiting on your review (writing from bug scrub)

Actions #15

Updated by Sridhar Seshasayee about 2 months ago

/a/skanta-2026-01-27_05:35:03-rados-wip-bharath1-testing-2026-01-26-1242-distro-default-trial
7 Jobs: ['19774', '19784', '19760', '19773', '19750', '19738', '19746']

Actions #16

Updated by Laura Flores about 2 months ago

/a/skanta-2026-01-28_09:27:17-rados-wip-bharath2-testing-2026-01-27-0639-squid-distro-default-trial/23106

Actions #17

Updated by Nitzan Mordechai about 2 months ago

/a/yuriw-2026-01-29_18:33:05-rados-wip-yuri2-testing-2026-01-28-1643-tentacle-distro-default-trial/
['26497', '26509', '26521', '26503', '26501', '26516', '26515']

Actions #18

Updated by Aishwarya Mathuria about 2 months ago

/a/skanta-2026-01-30_23:46:16-rados-wip-bharath7-testing-2026-01-29-2016-distro-default-trial
['28553', '28578', '28570', '28563', '28589', '28558', '28579']

Actions #19

Updated by Connor Fawcett about 1 month ago

/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19848
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19871
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19884
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19841
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19861
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19852
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19870

Actions #20

Updated by Kamoltat (Junior) Sirivadhna about 1 month ago

/a/skanta-2026-01-29_13:10:13-rados-wip-bharath6-testing-2026-01-29-0855-squid-distro-default-trial/
[25748, 25762]

Actions #21

Updated by Nitzan Mordechai about 1 month ago

/a/yuriw-2026-02-04_23:08:40-rados-wip-yuri3-testing-2026-02-04-1948-tentacle-distro-default-trial/
['35460', '35518', '35338', '35324', '35274', '35397', '35456']

Actions #22

Updated by Aishwarya Mathuria about 1 month ago

/a/skanta-2026-02-07_00:02:26-rados-wip-bharath7-testing-2026-02-06-0906-distro-default-trial
['39115', '39136', '39109', '39126', '39103', '39125', '39111']

Actions #23

Updated by Lee Sanders about 1 month ago

/a/skanta-2026-01-29_02:19:11-rados-wip-bharath5-testing-2026-01-28-2018-distro-default-trial/
[24577, 24821, 24642, 24700, 24764, 24626, 24760]

Actions #24

Updated by Lee Sanders about 1 month ago

/a/skanta-2026-01-29_13:05:02-rados-wip-bharath5-testing-2026-01-28-2018-distro-default-trial/
['25697', '25726', '25724', '25735', '25714', '25702', '25706']

Actions #25

Updated by Jaya Prakash about 1 month ago

/a/jayaprakash-2026-02-06_12:54:34-rados-jaya-bs-testing-05-02-2026-distro-default-trial
3 jobs: ['38153', '38053', '38082']

Actions #26

Updated by Nitzan Mordechai about 1 month ago

/a/yaarit-2026-02-08_02:25:21-rados-wip-rocky10-branch-of-the-day-2026-02-06-1770413686-distro-default-trial/39952
/a/yaarit-2026-02-08_02:25:21-rados-wip-rocky10-branch-of-the-day-2026-02-06-1770413686-distro-default-trial/39969

Actions #28

Updated by Nitzan Mordechai about 1 month ago

/a/nmordech-2026-02-09_08:34:53-rados-wip-rocky10-branch-of-the-day-2026-02-06-1770413686-distro-default-trial/
4 jobs: ['41155', '41289', '41138', '41285']

Actions #29

Updated by Aishwarya Mathuria about 1 month ago

/a/skanta-2026-02-05_03:38:32-rados-wip-bharath2-testing-2026-02-03-0542-distro-default-trial
['35662', '35661', '35637', '35641', '35645', '35671', '35653']

Actions #30

Updated by Laura Flores about 1 month ago

/a/yuriw-2026-01-27_16:01:09-rados-reef-release-distro-default-trial/21120

Actions #31

Updated by Naveen Naidu about 1 month ago

/a/skanta-2026-01-26_08:54:40-rados-wip-bharath4-testing-2026-01-26-1300-distro-default-trial/
7 jobs: ['17676', '17871', '17627', '17814', '17692', '17750', '17810']

Actions #32

Updated by Nitzan Mordechai about 1 month ago

/a/skanta-2026-02-02_23:43:28-rados-wip-bharath9-testing-2026-02-02-0839-distro-default-trial/
['30397', '30422', '30423', '30402', '30413', '30406', '30433']

Actions #33

Updated by Nitzan Mordechai about 1 month ago

/a/nmordech-2026-02-17_05:46:21-rados-wip-rocky10-branch-of-the-day-2026-02-13-1771001916-distro-default-trial/
['51772', '51923']

Actions #34

Updated by Radoslaw Zarzynski about 1 month ago

Still in QA.

Actions #35

Updated by Aishwarya Mathuria about 1 month ago

/a/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/
['53863', '53889']

Actions #36

Updated by Sridhar Seshasayee 26 days ago

/a/skanta-2026-02-22_05:18:48-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/
['63343', '63355']

Actions #37

Updated by Lee Sanders 26 days ago

/a/skanta-2026-02-07_14:54:11-rados-wip-bharath5-testing-2026-02-06-2052-distro-default-trial/
['39479', '39469', '39486', '39495', '39487', '39465', '39472']

Actions #39

Updated by Jaya Prakash 25 days ago

7 jobs: ['62239', '62045', '62059', '61995', '62181', '62177', '62118']
/a/yuriw-2026-02-21_00:21:18-rados-wip-yuri3-testing-2026-02-19-1610-tentacle-distro-default-trial

Actions #40

Updated by Laura Flores 23 days ago

/a/nmordech-2026-02-25_11:36:23-rados-wip-rocky10-branch-of-the-day-2026-02-24-1771941190-distro-default-trial/70133

Actions #41

Updated by Sridhar Seshasayee 19 days ago

/a/sseshasa-2026-02-26_14:56:45-rados-wip-sseshasa-testing-2026-02-26-1772100687-distro-default-trial
['72501', '72443', '72439', '72321', '72307', '72257', '72380']

Actions #42

Updated by Nitzan Mordechai 19 days ago

/a/yaarit-2026-02-26_20:20:34-rados-wip-rocky10-branch-of-the-day-2026-02-26-1772108951-distro-default-trial/
2 jobs: ['72874', '73025']

Actions #43

Updated by Radoslaw Zarzynski 19 days ago

Still in QA.

Actions #44

Updated by Nitzan Mordechai 16 days ago

/a/yaarit-2026-03-05_03:07:52-rados-wip-rocky10-branch-of-the-day-2026-03-04-1772624857-tentacle-distro-default-trial/
5 jobs: ['86616', '86746', '86601', '86482', '86750']

Actions #45

Updated by Laura Flores 16 days ago

/a/yaarit-2026-03-05_02:43:32-rados-wip-rocky10-branch-of-the-day-2026-03-04-1772633736-distro-default-trial/86388

Actions #46

Updated by Naveen Naidu 15 days ago

/a/yuriw-2026-03-02_18:34:01-rados-wip-yuri3-testing-2026-03-02-1622-distro-default-trial
2 jobs: ['76585', '76723']

Actions #47

Updated by Radoslaw Zarzynski 12 days ago

Bump up. Still in QA.

Actions #48

Updated by Aishwarya Mathuria 10 days ago

/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93046
/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93031
/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93057
/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93068
/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93019
['93055', '93035']

Actions #49

Updated by Connor Fawcett 9 days ago

/a/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68642
/a/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68633

Actions #50

Updated by Sridhar Seshasayee 8 days ago

/a/skanta-2026-03-04_23:53:38-rados-wip-bharath1-testing-2026-03-04-1011-distro-default-trial/
2 jobs: ['85640', '85621']

Actions #51

Updated by Jaya Prakash 5 days ago

/a/jayaprakash-2026-03-06_10:20:34-rados-wip-jaya-bs-testing-06-03-2025-distro-default-trial/90402

Actions #53

Updated by Lee Sanders 2 days ago

/a/pulpito.ceph.com/yuriw-2026-02-26_01:51:16-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/

['71045, '70910' ]

/a/yuriw-2026-02-26_16:03:22-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/
['72537','72552']

Actions

Also available in: Atom PDF