Project

General

Profile

Actions
Copy link

Bug #68756

closed

[notification]: segmentation fault in RGWPSListTopicsOp::execute()

Added by Shilpa MJ over 1 year ago. Updated 6 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Yuval Lifshitz
Target version:
-
% Done:

0%

Source:
Backport:
Regression:
Yes
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
60674
Tags (freeform):
Merge Commit:
862104bb09dfc8affae21e872a8503900a439e58
Fixed In:
v19.3.0-6079-g862104bb09
Released In:
v20.2.0~1616
Upkeep Timestamp:
2025-11-01T01:15:32+00:00

Description

in https://qa-proxy.ceph.com/teuthology/smanjara-2024-10-26_06:44:03-rgw:multisite-main-distro-default-smithi/7967467/teuthology.log

2024-10-26T07:38:04.910 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2
2024-10-26T07:38:04.914 INFO:tasks.rgw.c1.client.0.smithi044.stdout:*** Caught signal (Segmentation fault) *
2024-10-26T07:38:04.914 INFO:tasks.rgw.c1.client.0.smithi044.stdout: in thread 7fe358223640 thread_name:io_context_pool
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: ceph version 19.3.0-5762-gbd0160de (bd0160de81e216e42d835a3d4ce920c3bef81b16) squid (dev)
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fe3f4336520]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 2: (RGWPSListTopicsOp::execute(optional_yield)+0x140) [0x560d0ad4afe0]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 3: (rgw_process_authenticated(RGWHandler_REST, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0x9da) [0x560d0ab47b9a]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 4: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >, int*)+0x21d3) [0x560d0ab4a513]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 5: radosgw(+0x15416e0) [0x560d0b9a16e0]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 6: radosgw(+0x62e582) [0x560d0aa8e582]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 7: radosgw(+0x62e7a4) [0x560d0aa8e7a4]
2024-10-26T07:38:04.919 INFO:tasks.rgw.c1.client.0.smithi044.stdout: 8: make_fcontext()

Actions
Copy link
 #1

Updated by J. Eric Ivancich over 1 year ago

  • Priority changed from Normal to Urgent
Actions
Copy link
 #2

Updated by Casey Bodley over 1 year ago

crashing in s3tests also. from http://qa-proxy.ceph.com/teuthology/cbodley-2024-11-01_14:58:14-rgw-pr-55259-distro-default-smithi/7976109/teuthology.log:

2024-11-01T16:17:24.291 INFO:teuthology.orchestra.run.smithi103.stdout:s3tests_boto3/functional/test_s3select.py::test_progress_expressions PASSED [ 99%]
2024-11-01T16:17:24.624 INFO:teuthology.orchestra.run.smithi103.stdout:s3tests_boto3/functional/test_s3select.py::test_output_serial_expressions PASSED [ 99%]
2024-11-01T16:17:24.725 INFO:tasks.rgw.client.0.smithi103.stdout:*** Caught signal (Segmentation fault) **
2024-11-01T16:17:24.725 INFO:tasks.rgw.client.0.smithi103.stdout: in thread 7facbc969640 thread_name:io_context_pool
2024-11-01T16:17:24.725 INFO:tasks.rgw.client.0.smithi103.stdout: ceph version 19.3.0-5890-gc2576492 (c2576492b122dd86586593db9fc6e27197584094) squid (dev)
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fad7e63e6f0]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 2: (RGWPSListTopicsOp::execute(optional_yield)+0x13f) [0x55870d4efeaf]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 3: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0xa70) [0x55870d2e0b60]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 4: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0xfc1) [0x55870d2e2331]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 5: radosgw(+0x48fa40) [0x55870d1fda40]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 6: radosgw(+0x4b26a5) [0x55870d2206a5]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 7: radosgw(+0x496a84) [0x55870d204a84]
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: 8: make_fcontext()
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout:2024-11-01T16:17:24.723+0000 7facbc969640 -1 *** Caught signal (Segmentation fault) **
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: in thread 7facbc969640 thread_name:io_context_pool
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout:
2024-11-01T16:17:24.726 INFO:tasks.rgw.client.0.smithi103.stdout: ceph version 19.3.0-5890-gc2576492 (c2576492b122dd86586593db9fc6e27197584094) squid (dev)
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fad7e63e6f0]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 2: (RGWPSListTopicsOp::execute(optional_yield)+0x13f) [0x55870d4efeaf]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 3: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0xa70) [0x55870d2e0b60]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 4: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0xfc1) [0x55870d2e2331]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 5: radosgw(+0x48fa40) [0x55870d1fda40]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 6: radosgw(+0x4b26a5) [0x55870d2206a5]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 7: radosgw(+0x496a84) [0x55870d204a84]
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: 8: make_fcontext()
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2024-11-01T16:17:24.727 INFO:tasks.rgw.client.0.smithi103.stdout:
2024-11-01T16:17:24.735 INFO:tasks.rgw.client.0.smithi103.stdout:     0> 2024-11-01T16:17:24.723+0000 7facbc969640 -1 *** Caught signal (Segmentation fault) **
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: in thread 7facbc969640 thread_name:io_context_pool
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout:
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: ceph version 19.3.0-5890-gc2576492 (c2576492b122dd86586593db9fc6e27197584094) squid (dev)
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fad7e63e6f0]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 2: (RGWPSListTopicsOp::execute(optional_yield)+0x13f) [0x55870d4efeaf]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 3: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0xa70) [0x55870d2e0b60]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 4: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0xfc1) [0x55870d2e2331]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 5: radosgw(+0x48fa40) [0x55870d1fda40]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 6: radosgw(+0x4b26a5) [0x55870d2206a5]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 7: radosgw(+0x496a84) [0x55870d204a84]
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: 8: make_fcontext()
2024-11-01T16:17:24.736 INFO:tasks.rgw.client.0.smithi103.stdout: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2024-11-01T16:17:24.737 INFO:tasks.rgw.client.0.smithi103.stdout:
2024-11-01T16:17:24.757 INFO:tasks.rgw.client.0.smithi103.stdout:    -2> 2024-11-01T16:17:24.723+0000 7facbc969640 -1 *** Caught signal (Segmentation fault) **
2024-11-01T16:17:24.757 INFO:tasks.rgw.client.0.smithi103.stdout: in thread 7facbc969640 thread_name:io_context_pool
2024-11-01T16:17:24.757 INFO:tasks.rgw.client.0.smithi103.stdout:
2024-11-01T16:17:24.757 INFO:tasks.rgw.client.0.smithi103.stdout: ceph version 19.3.0-5890-gc2576492 (c2576492b122dd86586593db9fc6e27197584094) squid (dev)
2024-11-01T16:17:24.757 INFO:tasks.rgw.client.0.smithi103.stdout: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fad7e63e6f0]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 2: (RGWPSListTopicsOp::execute(optional_yield)+0x13f) [0x55870d4efeaf]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 3: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, rgw::sal::Driver*, bool)+0xa70) [0x55870d2e0b60]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 4: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0xfc1) [0x55870d2e2331]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 5: radosgw(+0x48fa40) [0x55870d1fda40]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 6: radosgw(+0x4b26a5) [0x55870d2206a5]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 7: radosgw(+0x496a84) [0x55870d204a84]
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: 8: make_fcontext()
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2024-11-01T16:17:24.758 INFO:tasks.rgw.client.0.smithi103.stdout:
2024-11-01T16:17:24.880 INFO:tasks.rgw.client.0.smithi103.stderr:daemon-helper: command crashed with signal 11
2024-11-01T16:17:28.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s
2024-11-01T16:17:34.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~6s
2024-11-01T16:17:40.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~13s
2024-11-01T16:17:47.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~19s
2024-11-01T16:17:53.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~25s
2024-11-01T16:17:53.653 INFO:teuthology.orchestra.run.smithi103.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic FAILED          [ 99%]

Actions
Copy link
 #3

Updated by Casey Bodley over 1 year ago

  • Subject changed from [rgw-multisite/notification]: segmentation fault while running test_topic_notification_sync() to [notification]: segmentation fault in RGWPSListTopicsOp::execute()
Actions
Copy link
 #4

Updated by Casey Bodley over 1 year ago

  • Status changed from New to Triaged

discussed with Yuval. this is a regression from https://github.com/ceph/ceph/pull/60045, which calls s->bucket->get_tenant() on an empty s->bucket (the pubsub topic APIs don't operate on buckets). the pr merged with a successful run of the rgw/notifications suite, but crashes in the rgw/verify and rgw/multisite suites

Actions
Copy link
 #5

Updated by Casey Bodley over 1 year ago

  • Status changed from Triaged to Fix Under Review
  • Regression changed from No to Yes
  • Pull request ID set to 60674
Actions
Copy link
 #6

Updated by Casey Bodley over 1 year ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link
 #8

Updated by Upkeep Bot 10 months ago

  • Merge Commit set to 862104bb09dfc8affae21e872a8503900a439e58
  • Fixed In set to v19.3.0-6079-g862104bb09d
  • Upkeep Timestamp set to 2025-07-10T08:44:15+00:00
Actions
Copy link
 #9

Updated by Upkeep Bot 10 months ago

  • Fixed In changed from v19.3.0-6079-g862104bb09d to v19.3.0-6079-g862104bb09
  • Upkeep Timestamp changed from 2025-07-10T08:44:15+00:00 to 2025-07-14T19:38:44+00:00
Actions
Copy link
 #10

Updated by Upkeep Bot 6 months ago

  • Released In set to v20.2.0~1616
  • Upkeep Timestamp changed from 2025-07-14T19:38:44+00:00 to 2025-11-01T01:15:32+00:00
Actions
Copy link

Also available in: Atom PDF