Bug #70707
closedmds.a asok_command: dump stray {prefix=dump stray} asserts MDS
0%
Description
-4> 2025-03-26T07:05:39.949+0000 7f5748da1640 1 mds.a asok_command: dump loads {prefix=dump loads} (starting...)
-3> 2025-03-26T07:05:39.949+0000 7f5748da1640 1 mds.a asok_command: dump snaps {prefix=dump snaps} (starting...)
-2> 2025-03-26T07:05:39.949+0000 7f5748da1640 1 mds.a asok_command: dump stray {prefix=dump stray} (starting...)
-1> 2025-03-26T07:05:39.953+0000 7f5748da1640 -1 ./src/common/Formatter.cc: In function 'virtual void ceph::JSONFormatter::close_section()' thread 7f5748da1640 time 2025-03-26T07:05:39.955480+0000
./src/common/Formatter.cc: 277: FAILED ceph_assert(!m_stack.empty())
ceph version 20.0.0-754-gb910a5cc (b910a5cc35f98a492c38ce539136f262f57cc549) tentacle (dev)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x117) [0x7f574cedf6a3]
2: /usr/lib/ceph/libceph-common.so.2(+0x186858) [0x7f574cedf858]
3: /usr/lib/ceph/libceph-common.so.2(+0x17ee46) [0x7f574ced7e46]
4: (MDCache::scan_stray_dir(dirfrag_t, std::unique_ptr<MDCache::C_MDS_DumpStrayDirCtx, std::default_delete<MDCache::C_MDS_DumpStrayDirCtx> >)+0xca) [0x5603a0b176ea]
5: (MDCache::stray_status(std::unique_ptr<MDCache::C_MDS_DumpStrayDirCtx, std::default_delete<MDCache::C_MDS_DumpStrayDirCtx> >)+0x36) [0x5603a0b17da6]
6: (MDSRankDispatcher::handle_asok_command(std::basic_string_view<char, std::char_traits<char> >, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&, ceph::Formatter*, ceph::buffer::v15_2_0::list const&, std::function<void (int, std::basic_string_view<char, std::char_traits<char> >, ceph::buffer::v15_2_0::list&)>)+0x3408) [0x5603a09a8068]
Most likely caused by:
https://github.com/ceph/ceph/pull/60684
Updated by Laura Flores 12 months ago · Edited
/a/yuriw-2025-03-27_15:03:25-rados-wip-yuri7-testing-2025-03-26-1605-distro-default-smithi/8213289
/a/yuriw-2025-03-27_15:03:25-rados-wip-yuri7-testing-2025-03-26-1605-distro-default-smithi/8213424
Updated by Igor Golikov 12 months ago
- Status changed from New to Fix Under Review
- Pull request ID set to 62578
Updated by Sridhar Seshasayee 12 months ago
/a/skanta-2025-03-27_08:02:07-rados-wip-bharath10-testing-2025-03-27-0430-distro-default-smithi/8212813
Updated by Aishwarya Mathuria 12 months ago
- Tags set to main-failures
/a//teuthology-2025-04-06_20:00:14-rados-main-distro-default-smithi/8226460
Updated by Laura Flores 12 months ago
/a/skanta-2025-04-04_06:10:17-rados-wip-bharath10-testing-2025-04-03-2112-distro-default-smithi/8223710
Updated by Igor Golikov 12 months ago
There is open PR for this fix
https://github.com/ceph/ceph/pull/62578
Does the last runs with the failure include this fix?
Updated by Sridhar Seshasayee 12 months ago
/a/yuriw-2025-04-07_23:32:09-rados-wip-yuri13-testing-2025-04-07-1144-distro-default-smithi/8229275
The fix is yet to be merged in main.
Updated by Laura Flores 12 months ago · Edited
/a/teuthology-2025-04-06_20:00:14-rados-main-distro-default-smithi/8226460
/a/teuthology-2025-04-06_20:00:14-rados-main-distro-default-smithi/8226612
Updated by Kamoltat (Junior) Sirivadhna 11 months ago
/a/skanta-2025-04-09_05:31:19-rados-wip-bharath17-testing-2025-04-08-0602-distro-default-smithi/
[8233192, 8233201]
Updated by Laura Flores 11 months ago
/a/lflores-2025-04-11_15:58:32-rados-wip-lflores-testing-2025-04-10-2245-distro-default-smithi/8235746
Updated by Laura Flores 11 months ago
/a/lflores-2025-04-15_00:57:49-rados-wip-lflores-testing-5-2025-04-14-1635-distro-default-gibba/8241448
Updated by Kamoltat (Junior) Sirivadhna 11 months ago
/a/lflores-2025-04-14_14:00:23-rados-wip-lflores-testing-2-2025-04-11-1133-distro-default-smithi/8239412
Updated by Laura Flores 11 months ago
/a/lflores-2025-04-11_19:10:45-rados-wip-lflores-testing-3-2025-04-11-1140-distro-default-smithi/8236029
Updated by Kamoltat (Junior) Sirivadhna 11 months ago
lflores-2025-04-14_22:30:34-rados-wip-lflores-testing-2025-04-10-2245-distro-default-gibba/
8241116
8241124
Updated by Venky Shankar 11 months ago
- Category set to Correctness/Safety
- Status changed from Fix Under Review to Pending Backport
- Target version set to v20.0.0
- Source set to Q/A
- Backport set to reef,squid
Updated by Venky Shankar 11 months ago
- Related to Feature #56442: mds: build asok command to dump stray files and associated caps added
Updated by Venky Shankar 11 months ago
@Igor Golikov - please pull in this fix with backports for https://tracker.ceph.com/issues/56442
Updated by Igor Golikov 11 months ago
Updated by Upkeep Bot 11 months ago
- Copied to Backport #71008: squid: mds.a asok_command: dump stray {prefix=dump stray} asserts MDS added
Updated by Upkeep Bot 11 months ago
- Copied to Backport #71009: reef: mds.a asok_command: dump stray {prefix=dump stray} asserts MDS added
Updated by Sridhar Seshasayee 11 months ago
/a/skanta-2025-04-22_23:21:15-rados-wip-bharath1-testing-2025-04-21-0529-distro-default-smithi/
[8254488, 8254505]
Updated by Aishwarya Mathuria 11 months ago
/a/yuriw-2025-04-14_18:07:07-rados-wip-yuri10-testing-2025-04-08-0710-distro-default-smithi/8239955
/a/yuriw-2025-04-14_18:07:07-rados-wip-yuri10-testing-2025-04-08-0710-distro-default-smithi/8239968
Updated by Upkeep Bot 9 months ago
- Merge Commit set to 0f86e32274fe7f608989c8ceab536a415e5b79b5
- Fixed In set to v20.0.0-1536-g0f86e32274f
- Upkeep Timestamp set to 2025-07-08T14:47:05+00:00
Updated by Upkeep Bot 8 months ago
- Fixed In changed from v20.0.0-1536-g0f86e32274f to v20.0.0-1536-g0f86e32274fe
- Upkeep Timestamp changed from 2025-07-08T14:47:05+00:00 to 2025-07-14T15:21:12+00:00
Updated by Upkeep Bot 8 months ago
- Fixed In changed from v20.0.0-1536-g0f86e32274fe to v20.0.0-1536-g0f86e32274
- Upkeep Timestamp changed from 2025-07-14T15:21:12+00:00 to 2025-07-14T20:45:45+00:00
Updated by Upkeep Bot 5 months ago
- Released In set to v20.2.0~538
- Upkeep Timestamp changed from 2025-07-14T20:45:45+00:00 to 2025-11-01T01:00:03+00:00
Updated by Igor Golikov 2 months ago
- Status changed from Pending Backport to Resolved