Project

General

Profile

Actions

Bug #43274

closed

unittest_rbd_mirror: Exception: SegFault

Added by Sebastian Wagner over 6 years ago. Updated about 1 year ago.

Status:
Duplicate
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

Unfortunately, I don't know what exactly went wrong:

185/191 Test #184: unittest_rbd_mirror .....................***Exception: SegFault 11.74 sec
[==========] Running 279 tests from 34 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from TestMockImageMap
[ RUN      ] TestMockImageMap.SetLocalImages
seed 1526
[       OK ] TestMockImageMap.SetLocalImages (8 ms)
[ RUN      ] TestMockImageMap.AddRemoveLocalImage
[       OK ] TestMockImageMap.AddRemoveLocalImage (25 ms)
[ RUN      ] TestMockImageMap.AddRemoveRemoteImage
[       OK ] TestMockImageMap.AddRemoveRemoteImage (15 ms)
[ RUN      ] TestMockImageMap.AddRemoveRemoteImageDuplicateNotification
[       OK ] TestMockImageMap.AddRemoveRemoteImageDuplicateNotification (5 ms)
[ RUN      ] TestMockImageMap.AcquireImageErrorRetry
[       OK ] TestMockImageMap.AcquireImageErrorRetry (2 ms)
[ RUN      ] TestMockImageMap.RemoveRemoteAndLocalImage
[       OK ] TestMockImageMap.RemoveRemoteAndLocalImage (2 ms)
[ RUN      ] TestMockImageMap.AddInstance
[       OK ] TestMockImageMap.AddInstance (4 ms)
[ RUN      ] TestMockImageMap.RemoveInstance
[       OK ] TestMockImageMap.RemoveInstance (7 ms)
[ RUN      ] TestMockImageMap.AddInstancePingPongImageTest
[       OK ] TestMockImageMap.AddInstancePingPongImageTest (34 ms)
[ RUN      ] TestMockImageMap.RemoveInstanceWithRemoveImage
[       OK ] TestMockImageMap.RemoveInstanceWithRemoveImage (23 ms)
[ RUN      ] TestMockImageMap.AddErrorAndRemoveImage
[       OK ] TestMockImageMap.AddErrorAndRemoveImage (35 ms)
[ RUN      ] TestMockImageMap.MirrorUUIDUpdated
[       OK ] TestMockImageMap.MirrorUUIDUpdated (44 ms)
[ RUN      ] TestMockImageMap.RebalanceImageMap
[       OK ] TestMockImageMap.RebalanceImageMap (40 ms)
[----------] 13 tests from TestMockImageMap (244 ms total)

[----------] 14 tests from TestMockImageReplayer
[ RUN      ] TestMockImageReplayer.StartStop
Failed to load class: cas (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_cas.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_cas.so: undefined symbol: _Z13cls_has_chunkPvNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
Failed to load class: log (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_log.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_log.so: undefined symbol: _Z24cls_cxx_map_write_headerPvPN4ceph6buffer7v14_2_04listE
Failed to load class: rgw (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_rgw.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_rgw.so: undefined symbol: _Z19cls_current_versionPv
Failed to load class: user (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_user.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_user.so: undefined symbol: _Z24cls_cxx_map_write_headerPvPN4ceph6buffer7v14_2_04listE
[       OK ] TestMockImageReplayer.StartStop (317 ms)
[ RUN      ] TestMockImageReplayer.LocalImagePrimary
[       OK ] TestMockImageReplayer.LocalImagePrimary (146 ms)
[ RUN      ] TestMockImageReplayer.LocalImageDNE
[       OK ] TestMockImageReplayer.LocalImageDNE (196 ms)
[ RUN      ] TestMockImageReplayer.PrepareLocalImageError
[       OK ] TestMockImageReplayer.PrepareLocalImageError (194 ms)
[ RUN      ] TestMockImageReplayer.GetRemoteImageIdDNE
[       OK ] TestMockImageReplayer.GetRemoteImageIdDNE (174 ms)
[ RUN      ] TestMockImageReplayer.GetRemoteImageIdNonLinkedDNE
[       OK ] TestMockImageReplayer.GetRemoteImageIdNonLinkedDNE (224 ms)
[ RUN      ] TestMockImageReplayer.GetRemoteImageIdError
[       OK ] TestMockImageReplayer.GetRemoteImageIdError (228 ms)
[ RUN      ] TestMockImageReplayer.BootstrapError
[       OK ] TestMockImageReplayer.BootstrapError (154 ms)
[ RUN      ] TestMockImageReplayer.StopBeforeBootstrap
[       OK ] TestMockImageReplayer.StopBeforeBootstrap (215 ms)
[ RUN      ] TestMockImageReplayer.StartExternalReplayError
[       OK ] TestMockImageReplayer.StartExternalReplayError (152 ms)
[ RUN      ] TestMockImageReplayer.StopError
[       OK ] TestMockImageReplayer.StopError (169 ms)
[ RUN      ] TestMockImageReplayer.Replay
[       OK ] TestMockImageReplayer.Replay (177 ms)
[ RUN      ] TestMockImageReplayer.DecodeError
[       OK ] TestMockImageReplayer.DecodeError (157 ms)
[ RUN      ] TestMockImageReplayer.DelayedReplay
[       OK ] TestMockImageReplayer.DelayedReplay (2153 ms)
[----------] 14 tests from TestMockImageReplayer (4663 ms total)

[----------] 5 tests from TestMockImageSync
[ RUN      ] TestMockImageSync.SimpleSync
[       OK ] TestMockImageSync.SimpleSync (198 ms)
[ RUN      ] TestMockImageSync.RestartSync
[       OK ] TestMockImageSync.RestartSync (173 ms)
[ RUN      ] TestMockImageSync.CancelNotifySyncRequest
[       OK ] TestMockImageSync.CancelNotifySyncRequest (159 ms)
[ RUN      ] TestMockImageSync.CancelImageCopy
[       OK ] TestMockImageSync.CancelImageCopy (195 ms)
[ RUN      ] TestMockImageSync.CancelAfterCopyImage
[       OK ] TestMockImageSync.CancelAfterCopyImage (166 ms)
[----------] 5 tests from TestMockImageSync (898 ms total)

[----------] 3 tests from TestMockInstanceReplayer
[ RUN      ] TestMockInstanceReplayer.AcquireReleaseImage
[       OK ] TestMockInstanceReplayer.AcquireReleaseImage (16 ms)
[ RUN      ] TestMockInstanceReplayer.RemoveFinishedImage
[       OK ] TestMockInstanceReplayer.RemoveFinishedImage (24 ms)
[ RUN      ] TestMockInstanceReplayer.Reacquire
[       OK ] TestMockInstanceReplayer.Reacquire (2 ms)
[----------] 3 tests from TestMockInstanceReplayer (42 ms total)

[----------] 11 tests from TestMockInstanceWatcher
[ RUN      ] TestMockInstanceWatcher.InitShutdown
[       OK ] TestMockInstanceWatcher.InitShutdown (23 ms)
[ RUN      ] TestMockInstanceWatcher.InitError
[       OK ] TestMockInstanceWatcher.InitError (18 ms)
[ RUN      ] TestMockInstanceWatcher.ShutdownError
[       OK ] TestMockInstanceWatcher.ShutdownError (15 ms)
[ RUN      ] TestMockInstanceWatcher.Remove
[       OK ] TestMockInstanceWatcher.Remove (16 ms)
[ RUN      ] TestMockInstanceWatcher.RemoveNoent
[       OK ] TestMockInstanceWatcher.RemoveNoent (12 ms)
[ RUN      ] TestMockInstanceWatcher.ImageAcquireRelease
[       OK ] TestMockInstanceWatcher.ImageAcquireRelease (36 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageRemoved
[       OK ] TestMockInstanceWatcher.PeerImageRemoved (36 ms)
[ RUN      ] TestMockInstanceWatcher.ImageAcquireReleaseCancel
[       OK ] TestMockInstanceWatcher.ImageAcquireReleaseCancel (31 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageAcquireWatchDNE
[       OK ] TestMockInstanceWatcher.PeerImageAcquireWatchDNE (17 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageReleaseWatchDNE
[       OK ] TestMockInstanceWatcher.PeerImageReleaseWatchDNE (32 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageRemovedCancel
[       OK ] TestMockInstanceWatcher.PeerImageRemovedCancel (12 ms)
[----------] 11 tests from TestMockInstanceWatcher (250 ms total)

[----------] 11 tests from TestMockInstanceWatcher_NotifySync
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartStopOnLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartStopOnLeader (48 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.CancelStartedOnLeader
[       OK ] TestMockInstanceWatcher_NotifySync.CancelStartedOnLeader (49 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartStopOnNonLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartStopOnNonLeader (36 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.CancelStartedOnNonLeader
[       OK ] TestMockInstanceWatcher_NotifySync.CancelStartedOnNonLeader (41 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.CancelWaitingOnNonLeader
[       OK ] TestMockInstanceWatcher_NotifySync.CancelWaitingOnNonLeader (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.InFlightPrevNotification
[       OK ] TestMockInstanceWatcher_NotifySync.InFlightPrevNotification (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.NoInFlightReleaseAcquireLeader
[       OK ] TestMockInstanceWatcher_NotifySync.NoInFlightReleaseAcquireLeader (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartedOnLeaderReleaseLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartedOnLeaderReleaseLeader (34 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.WaitingOnLeaderReleaseLeader
[       OK ] TestMockInstanceWatcher_NotifySync.WaitingOnLeaderReleaseLeader (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartedOnNonLeaderAcquireLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartedOnNonLeaderAcquireLeader (29 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.WaitingOnNonLeaderAcquireLeader
[       OK ] TestMockInstanceWatcher_NotifySync.WaitingOnNonLeaderAcquireLeader (34 ms)
[----------] 11 tests from TestMockInstanceWatcher_NotifySync (456 ms total)

[----------] 4 tests from TestMockLeaderWatcher
[ RUN      ] TestMockLeaderWatcher.InitShutdown
[       OK ] TestMockLeaderWatcher.InitShutdown (33 ms)
[ RUN      ] TestMockLeaderWatcher.InitReleaseShutdown
[       OK ] TestMockLeaderWatcher.InitReleaseShutdown (19 ms)
[ RUN      ] TestMockLeaderWatcher.AcquireError
[       OK ] TestMockLeaderWatcher.AcquireError (12 ms)
[ RUN      ] TestMockLeaderWatcher.Break
[       OK ] TestMockLeaderWatcher.Break (2012 ms)
[----------] 4 tests from TestMockLeaderWatcher (2076 ms total)

[----------] 12 tests from TestMockMirrorStatusUpdater
[ RUN      ] TestMockMirrorStatusUpdater.InitShutDown
[       OK ] TestMockMirrorStatusUpdater.InitShutDown (13 ms)
[ RUN      ] TestMockMirrorStatusUpdater.InitStatusWatcherError
[       OK ] TestMockMirrorStatusUpdater.InitStatusWatcherError (26 ms)
[ RUN      ] TestMockMirrorStatusUpdater.ShutDownStatusWatcherError
[       OK ] TestMockMirrorStatusUpdater.ShutDownStatusWatcherError (14 ms)
[ RUN      ] TestMockMirrorStatusUpdater.SmallBatch
[       OK ] TestMockMirrorStatusUpdater.SmallBatch (24 ms)
[ RUN      ] TestMockMirrorStatusUpdater.LargeBatch
[       OK ] TestMockMirrorStatusUpdater.LargeBatch (30 ms)
[ RUN      ] TestMockMirrorStatusUpdater.OverwriteStatus
[       OK ] TestMockMirrorStatusUpdater.OverwriteStatus (11 ms)
[ RUN      ] TestMockMirrorStatusUpdater.OverwriteStatusInFlight
[       OK ] TestMockMirrorStatusUpdater.OverwriteStatusInFlight (7 ms)
[ RUN      ] TestMockMirrorStatusUpdater.ImmediateUpdate
[       OK ] TestMockMirrorStatusUpdater.ImmediateUpdate (9 ms)
[ RUN      ] TestMockMirrorStatusUpdater.RemoveIdleStatus
[       OK ] TestMockMirrorStatusUpdater.RemoveIdleStatus (20 ms)
[ RUN      ] TestMockMirrorStatusUpdater.RemoveInFlightStatus
[       OK ] TestMockMirrorStatusUpdater.RemoveInFlightStatus (9 ms)
[ RUN      ] TestMockMirrorStatusUpdater.ShutDownWhileUpdating
[       OK ] TestMockMirrorStatusUpdater.ShutDownWhileUpdating (14 ms)
[ RUN      ] TestMockMirrorStatusUpdater.MirrorPeerSitePing
[       OK ] TestMockMirrorStatusUpdater.MirrorPeerSitePing (24 ms)
[----------] 12 tests from TestMockMirrorStatusUpdater (201 ms total)

[----------] 6 tests from TestMockNamespaceReplayer
[ RUN      ] TestMockNamespaceReplayer.Init_LocalMirrorStatusUpdaterError
[       OK ] TestMockNamespaceReplayer.Init_LocalMirrorStatusUpdaterError (55 ms)
[ RUN      ] TestMockNamespaceReplayer.Init_RemoteMirrorStatusUpdaterError
[       OK ] TestMockNamespaceReplayer.Init_RemoteMirrorStatusUpdaterError (32 ms)
[ RUN      ] TestMockNamespaceReplayer.Init_InstanceReplayerError
[       OK ] TestMockNamespaceReplayer.Init_InstanceReplayerError (12 ms)
[ RUN      ] TestMockNamespaceReplayer.Init_InstanceWatcherError
[       OK ] TestMockNamespaceReplayer.Init_InstanceWatcherError (20 ms)
[ RUN      ] TestMockNamespaceReplayer.Init
[       OK ] TestMockNamespaceReplayer.Init (16 ms)
[ RUN      ] TestMockNamespaceReplayer.AcuqireLeader
[       OK ] TestMockNamespaceReplayer.AcuqireLeader (9 ms)
[----------] 6 tests from TestMockNamespaceReplayer (144 ms total)

[----------] 4 tests from TestMockPoolReplayer
[ RUN      ] TestMockPoolReplayer.ConfigKeyOverride
[       OK ] TestMockPoolReplayer.ConfigKeyOverride (47 ms)
[ RUN      ] TestMockPoolReplayer.AcquireReleaseLeader
[       OK ] TestMockPoolReplayer.AcquireReleaseLeader (55 ms)
[ RUN      ] TestMockPoolReplayer.Namespaces
[       OK ] TestMockPoolReplayer.Namespaces (2075 ms)
[ RUN      ] TestMockPoolReplayer.NamespacesError

https://jenkins.ceph.com/job/ceph-pull-requests/40443/console

https://github.com/ceph/ceph/pull/32182


Related issues 3 (0 open3 closed)

Related to CI - Bug #43566: jenkins: ctest output size for failed test is too shortResolvedDavid Galloway

Actions
Related to rbd - Bug #46679: make check: run-rbd-unit-tests-127.sh: TestLibRBD.QuiesceWatchError: pure virtual method calledResolvedJason Dillaman

Actions
Is duplicate of rbd - Bug #63798: make check: unittest_rbd_mirror SEGFAULTResolvedIlya Dryomov

Actions
Actions #1

Updated by Jason Dillaman about 6 years ago

  • Status changed from New to Need More Info
Actions #2

Updated by Mykola Golub about 6 years ago

Reported about the issue with truncated failed test output to CI project [1].

[1] https://tracker.ceph.com/issues/43566

Actions #3

Updated by Sebastian Wagner over 5 years ago

  • Related to Bug #43566: jenkins: ctest output size for failed test is too short added
Actions #4

Updated by Sebastian Wagner over 5 years ago

  • Related to Bug #46679: make check: run-rbd-unit-tests-127.sh: TestLibRBD.QuiesceWatchError: pure virtual method called added
Actions #5

Updated by Kefu Chai almost 5 years ago

[ RUN      ] TestImageReplayer/4.SnapshotLimit
remote_snap_id=4
local_snap_id=3, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=4, last_copied_object_number=0, snap_seqs={4=18446744073709551614}]
flushed
remote_snap_id=5
local_snap_id=4, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=5, last_copied_object_number=0, snap_seqs={5=18446744073709551614}]
flushed
remote_snap_id=6
local_snap_id=5, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=6, last_copied_object_number=0, snap_seqs={6=18446744073709551614}]

....
[ RUN      ] TestImageReplayer/4.SnapshotLimit
remote_snap_id=4
local_snap_id=3, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=4, last_copied_object_number=0, snap_seqs={4=18446744073709551614}]
flushed
remote_snap_id=5
local_snap_id=4, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=5, last_copied_object_number=0, snap_seqs={5=18446744073709551614}]
flushed
remote_snap_id=6
local_snap_id=5, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=6, last_copied_object_number=0, snap_seqs={6=18446744073709551614}]
...
The following tests FAILED:
    205 - unittest_rbd_mirror (SEGFAULT)
Actions #6

Updated by Deepika Upadhyay over 4 years ago

 [----------] 4 tests from TestMockPoolReplayer
  3287 [ RUN      ] TestMockPoolReplayer.ConfigKeyOverride
  3288 terminate called after throwing an instance of 'std::out_of_range'
  3289   what():  bitset::test: __position (which is 7955819) >= _Nb (which is 131072)

locally rep

Actions #7

Updated by Casey Bodley over 1 year ago

this is old so maybe not the same thing, but this happens occasionally in make check:

The following tests FAILED:
    247 - unittest_rbd_mirror (SEGFAULT)

ex https://jenkins.ceph.com/job/ceph-pull-requests/142284/

Actions #8

Updated by Ilya Dryomov over 1 year ago

Looks the same as https://tracker.ceph.com/issues/43274#note-5 to me. This ticket is old, but I don't think anything changed wrt. this test in the past couple of years.

Actions #9

Updated by Casey Bodley over 1 year ago

again from https://jenkins.ceph.com/job/ceph-pull-requests-arm64/65526/

The following tests FAILED:
    246 - unittest_rbd_mirror (SEGFAULT)
Actions #10

Updated by Casey Bodley about 1 year ago

https://jenkins.ceph.com/job/ceph-pull-requests/149102/ (but i don't see the actual segfault in the log)

The following tests FAILED:
    248 - unittest_rbd_mirror (SEGFAULT)

Actions #11

Updated by Ilya Dryomov about 1 year ago

  • Status changed from Need More Info to Duplicate
  • Assignee set to Ilya Dryomov
Actions #12

Updated by Ilya Dryomov about 1 year ago

  • Is duplicate of Bug #63798: make check: unittest_rbd_mirror SEGFAULT added
Actions #13

Updated by marcel lauhoff about 1 year ago

https://github.com/ceph/ceph/pull/61264 hits a segfault in unittest_rbd_mirror as well.

I could narrow it down a bit. I have no idea what the tests do and if running them selectively is ok, but running test NamespacesError before AcuireReleaseLeader hits an assertion all the time on my machine.

bin/unittest_rbd_mirror --gtest_shuffle --gtest_random_seed=85408 --gtest_filter='*NamespacesError*:*AcquireReleaseLeader*'
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from TestMockPoolReplayer
[ RUN      ] TestMockPoolReplayer.NamespacesError
seed 471498
Failed to load class: cas (lib/libcls_cas.so): lib/libcls_cas.so: undefined symbol: _Z26cls_get_manifest_ref_countPvNSt7__cxx1112basic_stringIcSt11char_t
raitsIcESaIcEEE
Failed to load class: cmpomap (lib/libcls_cmpomap.so): lib/libcls_cmpomap.so: undefined symbol: _Z28cls_cxx_map_get_vals_by_keysPvRKSt3setINSt7__cxx1112b
asic_stringIcSt11char_traitsIcESaIcEEESt4lessIS6_ESaIS6_EEPSt3mapIS6_N4ceph6buffer7v15_2_04listES8_SaISt4pairIKS6_SH_EEE
Failed to load class: fifo (lib/libcls_fifo.so): lib/libcls_fifo.so: undefined symbol: _Z20cls_gen_random_bytesPci
Failed to load class: log (lib/libcls_log.so): lib/libcls_log.so: undefined symbol: _Z24cls_cxx_map_write_headerPvPN4ceph6buffer7v15_2_04listE
Failed to load class: lua (lib/libcls_lua.so): lib/libcls_lua.so: undefined symbol: _Z19cls_current_versionPv
Failed to load class: rgw (lib/libcls_rgw.so): lib/libcls_rgw.so: undefined symbol: _Z19cls_get_object_infoPv
Failed to load class: user (lib/libcls_user.so): lib/libcls_user.so: undefined symbol: _Z24cls_cxx_map_write_headerPvPN4ceph6buffer7v15_2_04listE
[       OK ] TestMockPoolReplayer.NamespacesError (2064 ms)
[ RUN      ] TestMockPoolReplayer.AcquireReleaseLeader
/compile2/ceph/wip/src/common/mutex_debug.h: In function 'void ceph::mutex_debug_detail::mutex_debug_impl<false>::_post_lock()' thread 7fc388ff96c0 time
2025-02-18T11:02:54.972385+0100
/compile2/ceph/wip/src/common/mutex_debug.h: 154: FAILED ceph_assert(nlock == 0)
 ceph version Development (no_version) squid (dev)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x197) [0x7fc5643c3917]
 2: /compile2/ceph/wip/build/lib/libceph-common.so.2(+0x15c377f) [0x7fc5643c377f]
 3: (ceph::mutex_debug_detail::mutex_debug_impl<false>::_post_lock()+0x3b) [0x1ee174b]
 4: (ceph::mutex_debug_detail::mutex_debug_impl<false>::_try_lock(bool)+0x60) [0x1ee1610]
 5: (ceph::mutex_debug_detail::mutex_debug_impl<false>::lock(bool)+0x7d) [0x1eae89d]
 6: (std::lock_guard<ceph::mutex_debug_detail::mutex_debug_impl<false> >::lock_guard(ceph::mutex_debug_detail::mutex_debug_impl<false>&)+0x25) [0x1ea9935
]
 7: bin/unittest_rbd_mirror() [0x20de611]
 8: bin/unittest_rbd_mirror() [0x20de43e]
 9: bin/unittest_rbd_mirror() [0x20dce43]
 10: bin/unittest_rbd_mirror() [0x20dbf7a]
 11: bin/unittest_rbd_mirror() [0x20dbd68]
 12: bin/unittest_rbd_mirror() [0x20d76cf]
 13: bin/unittest_rbd_mirror() [0x20d6729]
 14: bin/unittest_rbd_mirror() [0x20d63a9]
 15: (Thread::entry_wrapper()+0x77) [0x7fc564374d07]
 16: (Thread::_entry_func(void*)+0x15) [0x7fc564374c75]
 17: /lib64/libc.so.6(+0x98088) [0x7fc5622a6088]
 18: /lib64/libc.so.6(+0x11bf8c) [0x7fc562329f8c]
Actions #14

Updated by Ilya Dryomov about 1 year ago

marcel lauhoff wrote in #note-13:

https://github.com/ceph/ceph/pull/61264 hits a segfault in unittest_rbd_mirror as well.

Hi Marcel,

The fix for this segfault has just landed in main via https://tracker.ceph.com/issues/63798.

I could narrow it down a bit. I have no idea what the tests do and if running them selectively is ok, but running test NamespacesError before AcuireReleaseLeader hits an assertion all the time on my machine.

This is a different issue. Running tests selectively is OK, but we don't do shuffled runs often. NamespacesError just doesn't clean up after itself, so running pretty much any test after it is a problem. I'll fix this under a new tracker.

Actions #15

Updated by Ilya Dryomov about 1 year ago

Ilya Dryomov wrote in #note-14:

I'll fix this under a new tracker.

https://tracker.ceph.com/issues/70041

Actions

Also available in: Atom PDF