Actions
Bug #64942
closedrados/verify: valgrind reports "Invalid read of size 8" error.
% Done:
0%
Source:
Backport:
reef,squid
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
v19.3.0-3845-ge991a822b5
Released In:
v20.2.0~2348
Upkeep Timestamp:
2025-11-01T01:19:44+00:00
Description
/a/yuriw-2024-03-08_16:20:46-rados-wip-yuri4-testing-2024-03-05-0854-distro-default-smithi/7587319
/a/yuriw-2024-03-08_16:20:46-rados-wip-yuri4-testing-2024-03-05-0854-distro-default-smithi/7587941
Error from osd.1 of JobID 7587319:
<error>
<unique>0x19392</unique>
<tid>70</tid>
<threadname>tp_osd_tp</threadname>
<kind>InvalidRead</kind>
<what>Invalid read of size 8</what>
<stack>
<frame>
<ip>0x5DAFC9</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>basic_string.h</file>
<line>195</line>
</frame>
<frame>
<ip>0x5DAFC9</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>basic_string.h</file>
<line>459</line>
</frame>
<frame>
<ip>0x5DAFC9</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/include</dir>
<file>object.h</file>
<line>37</line>
</frame>
<frame>
<ip>0x5DAFC9</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>hobject_t::hobject_t(hobject_t const&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>hobject.h</file>
<line>121</line>
</frame>
<frame>
<ip>0x734EBE</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>.LTHUNK127.lto_priv.0</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PrimaryLogPG.cc</file>
<line>470</line>
</frame>
<frame>
<ip>0x9E35D4</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ReplicatedBackend::handle_pull_response(pg_shard_t, PushOp const&, PullOp*, std::__cxx11::list<ReplicatedBackend::pull_complete_info, std::allocator<ReplicatedBackend::pull_complete_info> >*, ceph::os::Transaction*)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.cc</file>
<line>1905</line>
</frame>
<frame>
<ip>0x9E4330</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ReplicatedBackend::_do_pull_response(boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.cc</file>
<line>879</line>
</frame>
<frame>
<ip>0x9C7979</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.h</file>
<line>224</line>
</frame>
<frame>
<ip>0x9C7979</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ReplicatedBackend::_handle_message(boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.cc</file>
<line>211</line>
</frame>
<frame>
<ip>0x807165</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>PGBackend::handle_message(boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PGBackend.cc</file>
<line>131</line>
</frame>
<frame>
<ip>0x7509EC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PrimaryLogPG.cc</file>
<line>1897</line>
</frame>
<frame>
<ip>0x689F26</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>9742</line>
</frame>
<frame>
<ip>0x8D6537</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ceph::osd::scheduler::PGRecoveryMsg::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd/scheduler</dir>
<file>OpSchedulerItem.cc</file>
<line>241</line>
</frame>
<frame>
<ip>0x6A52C6</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd/scheduler</dir>
<file>OpSchedulerItem.h</file>
<line>144</line>
</frame>
<frame>
<ip>0x6A52C6</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>11182</line>
</frame>
<frame>
<ip>0xBA5D99</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ShardedThreadPool::shardedthreadpool_worker(unsigned int)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>WorkQueue.cc</file>
<line>313</line>
</frame>
<frame>
<ip>0xBA6343</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ShardedThreadPool::WorkThreadSharded::entry()</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>WorkQueue.h</file>
<line>650</line>
</frame>
<frame>
<ip>0x54EB801</ip>
<obj>/usr/lib64/libc.so.6</obj>
<fn>start_thread</fn>
</frame>
</stack>
<auxwhat>Address 0x18f65f20 is 32 bytes inside a block of size 184 free'd</auxwhat>
<stack>
<frame>
<ip>0x4848BC8</ip>
<obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
<fn>operator delete[](void*, unsigned long)</fn>
<dir>/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc</dir>
<file>vg_replace_malloc.c</file>
<line>1323</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/ext</dir>
<file>new_allocator.h</file>
<line>145</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>allocator.h</file>
<line>199</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>alloc_traits.h</file>
<line>496</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>565</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>632</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>2487</line>
</frame>
<frame>
<ip>0x7C441A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>std::_Rb_tree<hobject_t, std::pair<hobject_t const, std::__cxx11::list<boost::intrusive_ptr<OpRequest>, std::allocator<boost::intrusive_ptr<OpRequest> > > >, std::_Select1st<std::pair<hobject_t const, std::__cxx11::list<boost::intrusive_ptr<OpRequest>, std::allocator<boost::intrusive_ptr<OpRequest> > > > >, std::less<hobject_t>, std::allocator<std::pair<hobject_t const, std::__cxx11::list<boost::intrusive_ptr<OpRequest>, std::allocator<boost::intrusive_ptr<OpRequest> > > > > >::erase[abi:cxx11](std::_Rb_tree_iterator<std::pair<hobject_t const, std::__cxx11::list<boost::intrusive_ptr<OpRequest>, std::allocator<boost::intrusive_ptr<OpRequest> > > > >)</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>1209</line>
</frame>
<frame>
<ip>0x734EAA</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_map.h</file>
<line>1038</line>
</frame>
<frame>
<ip>0x734EAA</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>.LTHUNK127.lto_priv.0</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PrimaryLogPG.cc</file>
<line>469</line>
</frame>
<frame>
<ip>0x9E35D4</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ReplicatedBackend::handle_pull_response(pg_shard_t, PushOp const&, PullOp*, std::__cxx11::list<ReplicatedBackend::pull_complete_info, std::allocator<ReplicatedBackend::pull_complete_info> >*, ceph::os::Transaction*)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.cc</file>
<line>1905</line>
</frame>
<frame>
<ip>0x9E4330</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ReplicatedBackend::_do_pull_response(boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.cc</file>
<line>879</line>
</frame>
<frame>
<ip>0x9C7979</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.h</file>
<line>224</line>
</frame>
<frame>
<ip>0x9C7979</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ReplicatedBackend::_handle_message(boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>ReplicatedBackend.cc</file>
<line>211</line>
</frame>
<frame>
<ip>0x807165</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>PGBackend::handle_message(boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PGBackend.cc</file>
<line>131</line>
</frame>
<frame>
<ip>0x7509EC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PrimaryLogPG.cc</file>
<line>1897</line>
</frame>
<frame>
<ip>0x689F26</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>9742</line>
</frame>
<frame>
<ip>0x8D6537</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ceph::osd::scheduler::PGRecoveryMsg::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd/scheduler</dir>
<file>OpSchedulerItem.cc</file>
<line>241</line>
</frame>
<frame>
<ip>0x6A52C6</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd/scheduler</dir>
<file>OpSchedulerItem.h</file>
<line>144</line>
</frame>
<frame>
<ip>0x6A52C6</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>11182</line>
</frame>
<frame>
<ip>0xBA5D99</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ShardedThreadPool::shardedthreadpool_worker(unsigned int)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>WorkQueue.cc</file>
<line>313</line>
</frame>
<frame>
<ip>0xBA6343</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ShardedThreadPool::WorkThreadSharded::entry()</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>WorkQueue.h</file>
<line>650</line>
</frame>
<frame>
<ip>0x54EB801</ip>
<obj>/usr/lib64/libc.so.6</obj>
<fn>start_thread</fn>
</frame>
</stack>
<auxwhat>Block was alloc'd at</auxwhat>
<stack>
<frame>
<ip>0x48462F3</ip>
<obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
<fn>operator new[](unsigned long)</fn>
<dir>/builddir/build/BUILD/valgrind-3.21.0/coregrind/m_replacemalloc</dir>
<file>vg_replace_malloc.c</file>
<line>714</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/ext</dir>
<file>new_allocator.h</file>
<line>127</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>allocator.h</file>
<line>185</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>alloc_traits.h</file>
<line>464</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>561</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>611</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_tree.h</file>
<line>2431</line>
</frame>
<frame>
<ip>0x7C47AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>std::map<hobject_t, std::__cxx11::list<boost::intrusive_ptr<OpRequest>, std::allocator<boost::intrusive_ptr<OpRequest> > >, std::less<hobject_t>, std::allocator<std::pair<hobject_t const, std::__cxx11::list<boost::intrusive_ptr<OpRequest>, std::allocator<boost::intrusive_ptr<OpRequest> > > > > >::operator[](hobject_t const&)</fn>
<dir>/usr/include/c++/11/bits</dir>
<file>stl_map.h</file>
<line>501</line>
</frame>
<frame>
<ip>0x737460</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>PrimaryLogPG::wait_for_unreadable_object(hobject_t const&, boost::intrusive_ptr<OpRequest>)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PrimaryLogPG.cc</file>
<line>632</line>
</frame>
<frame>
<ip>0x74CFD1</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>PrimaryLogPG.cc</file>
<line>2186</line>
</frame>
<frame>
<ip>0x689F26</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>9742</line>
</frame>
<frame>
<ip>0x8D5998</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd/scheduler</dir>
<file>OpSchedulerItem.cc</file>
<line>33</line>
</frame>
<frame>
<ip>0x6A52C6</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>UnknownInlinedFun</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd/scheduler</dir>
<file>OpSchedulerItem.h</file>
<line>144</line>
</frame>
<frame>
<ip>0x6A52C6</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>11182</line>
</frame>
<frame>
<ip>0xBA5D99</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ShardedThreadPool::shardedthreadpool_worker(unsigned int)</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>WorkQueue.cc</file>
<line>313</line>
</frame>
<frame>
<ip>0xBA6343</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>ShardedThreadPool::WorkThreadSharded::entry()</fn>
<dir>/usr/src/debug/ceph-19.0.0-1875.g831bc188.el9.x86_64/src/common</dir>
<file>WorkQueue.h</file>
<line>650</line>
</frame>
<frame>
<ip>0x54EB801</ip>
<obj>/usr/lib64/libc.so.6</obj>
<fn>start_thread</fn>
</frame>
</stack>
</error>
Updated by Aishwarya Mathuria almost 2 years ago
/a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648773/
Updated by Nitzan Mordechai almost 2 years ago
- Status changed from New to Fix Under Review
- Assignee set to Nitzan Mordechai
- Pull request ID set to 57545
Updated by Laura Flores over 1 year ago
/a/lflores-2024-07-18_16:04:38-rados:verify-main-distro-default-smithi/7807288
Updated by Laura Flores over 1 year ago
QA review in progress here: https://tracker.ceph.com/issues/67027
Updated by Laura Flores over 1 year ago
- Status changed from Fix Under Review to Resolved
Updated by Upkeep Bot 8 months ago
- Merge Commit set to e991a822b577c9c34b4f42c02673918bbc53a724
- Fixed In set to v19.3.0-3845-ge991a822b57
- Upkeep Timestamp set to 2025-07-11T14:16:30+00:00
Updated by Upkeep Bot 8 months ago
- Fixed In changed from v19.3.0-3845-ge991a822b57 to v19.3.0-3845-ge991a822b5
- Upkeep Timestamp changed from 2025-07-11T14:16:30+00:00 to 2025-07-14T23:10:16+00:00
Updated by Upkeep Bot 5 months ago
- Released In set to v20.2.0~2348
- Upkeep Timestamp changed from 2025-07-14T23:10:16+00:00 to 2025-11-01T01:19:44+00:00
Actions