Project

General

Profile

Actions

Bug #68309

open

client: libcephfs aio crashes with assert "ceph_assert(len > 0)""

Added by Kotresh Hiremath Ravishankar over 1 year ago. Updated 5 months ago.

Status:
Pending Backport
Priority:
Normal
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Development
Backport:
squid
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, libcephfs
Labels (FS):
Pull request ID:
Tags (freeform):
backport_processed
Fixed In:
v19.3.0-5458-g43012083f4
Released In:
v20.2.0~1832
Upkeep Timestamp:
2025-11-01T01:02:09+00:00

Description

While running libcephfs client aio tests with object cacher disabled after fixing the issue https://tracker.ceph.com/issues/68146 and https://tracker.ceph.com/issues/68308, the client crashes
with the assert "ceph_assert(len > 0)"

Reproducer:
1. Included the fix for https://tracker.ceph.com/issues/68146 and https://tracker.ceph.com/issues/68308
2. Ran libcephfs client aio tests on vstart cluster as below
- bin/ceph_test_client --client_oc=false
3. TestClient.LlreadvLlwritevZeroBytes fails (src/test/client/nonblocking.cc)

Please find the traceback below

    ceph/src/osdc/Striper.cc: 186: FAILED ceph_assert(len > 0)

    Traceback:
     ceph version Development (no_version) squid (dev)
     1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x125) [0x7fc0a340aafe]
     2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc0a340ad20]
     3: (Striper::file_to_extents(ceph::common::CephContext*, file_layout_t const*, unsigned long, unsigned long, unsigned long, unsigned long, boost::container::small_vector<striper::Lightweight
ObjectExtent, 4ul, void, void>*)+0x184) [0x562727e13ab4]
     4: (Striper::file_to_extents(ceph::common::CephContext*, char const*, file_layout_t const*, unsigned long, unsigned long, unsigned long, std::vector<ObjectExtent, std::allocator<ObjectExtent
> >&, unsigned long)+0x97) [0x562727e145d1]
     5: (Striper::file_to_extents(ceph::common::CephContext*, inodeno_t, file_layout_t const*, unsigned long, unsigned long, unsigned long, std::vector<ObjectExtent, std::allocator<ObjectExtent> 
>&)+0x75) [0x562727d29520]
     6: (Filer::read_trunc(inodeno_t, file_layout_t const*, snapid_t, unsigned long, unsigned long, ceph::buffer::v15_2_0::list*, int, unsigned long, unsigned int, Context*, int)+0x61) [0x562727d66ea5]
     7: (Client::C_Read_Sync_NonBlocking::retry()+0x10c) [0x562727cd8a8e]
     8: (Client::_read(Fh*, long, unsigned long, ceph::buffer::v15_2_0::list*, Context*)+0x578) [0x562727d10cb6]
     9: (Client::_preadv_pwritev_locked(Fh*, iovec const*, int, long, bool, bool, Context*, ceph::buffer::v15_2_0::list*, bool, bool)+0x3a7) [0x562727d18159]
     10: (Client::ll_preadv_pwritev(Fh*, iovec const*, int, long, bool, Context*, ceph::buffer::v15_2_0::list*, bool, bool)+0x179) [0x562727d18b99]
     11: (TestClient_LlreadvLlwritevZeroBytes_Test::TestBody()+0x592) [0x562727ca5352]
     12: (void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x1b) [0x562727d9dea3]
     13: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x80) [0x562727da2b26]
     14: (testing::Test::Run()+0xb4) [0x562727d927ae]
     15: (testing::TestInfo::Run()+0x104) [0x562727d92988]
     16: (testing::TestSuite::Run()+0xb2) [0x562727d92b34]
     17: (testing::internal::UnitTestImpl::RunAllTests()+0x36b) [0x562727d95303]
     18: (bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x1b) [0x562727d9e15f]
     19: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x80) [0x562727da3083]
     20: (testing::UnitTest::Run()+0x63) [0x562727d92813]
     21: (RUN_ALL_TESTS()+0x11) [0x562727c828d9]
     22: main()


Related issues 2 (1 open1 closed)

Related to CephFS - Bug #68146: NFS Ganesha fsal CEPH fio denali workload causing "ms_dispatch" received signal SIGSEGV, Segmentation fault.ResolvedKotresh Hiremath Ravishankar

Actions
Copied to CephFS - Backport #68493: squid: client: libcephfs aio crashes with assert "ceph_assert(len > 0)""QA TestingKotresh Hiremath RavishankarActions
Actions #1

Updated by Kotresh Hiremath Ravishankar over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to Kotresh Hiremath Ravishankar
Actions #2

Updated by Kotresh Hiremath Ravishankar over 1 year ago

  • Related to Bug #68146: NFS Ganesha fsal CEPH fio denali workload causing "ms_dispatch" received signal SIGSEGV, Segmentation fault. added
Actions #3

Updated by Kotresh Hiremath Ravishankar over 1 year ago

  • Pull request ID set to 59987
Actions #4

Updated by Kotresh Hiremath Ravishankar over 1 year ago

  • Subject changed from client: libcephfs aio crashes with assert "cap_refs[c] > 0" to client: libcephfs aio crashes with assert "ceph_assert(len > 0)""
Actions #5

Updated by Kotresh Hiremath Ravishankar over 1 year ago

  • Status changed from In Progress to Fix Under Review
Actions #6

Updated by Venky Shankar over 1 year ago

  • Category set to Correctness/Safety
  • Status changed from Fix Under Review to Pending Backport
  • Target version set to v20.0.0
  • Source set to Development
  • Backport changed from quincy,reef,squid to squid
Actions #7

Updated by Upkeep Bot over 1 year ago

  • Copied to Backport #68493: squid: client: libcephfs aio crashes with assert "ceph_assert(len > 0)"" added
Actions #8

Updated by Upkeep Bot over 1 year ago

  • Tags (freeform) set to backport_processed
Actions #9

Updated by Upkeep Bot 9 months ago

  • Merge Commit set to 43012083f476f0a4d9b432fc2b51251d8c3efa8a
  • Fixed In set to v19.3.0-5458-g43012083f47
  • Upkeep Timestamp set to 2025-07-08T18:35:02+00:00
Actions #10

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-5458-g43012083f47 to v19.3.0-5458-g43012083f476
  • Upkeep Timestamp changed from 2025-07-08T18:35:02+00:00 to 2025-07-14T15:45:04+00:00
Actions #11

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-5458-g43012083f476 to v19.3.0-5458-g43012083f4
  • Upkeep Timestamp changed from 2025-07-14T15:45:04+00:00 to 2025-07-14T21:09:38+00:00
Actions #12

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~1832
  • Upkeep Timestamp changed from 2025-07-14T21:09:38+00:00 to 2025-11-01T01:02:09+00:00
Actions

Also available in: Atom PDF