Actions
Bug #64996
closedcrimson: crash in ScrubScan::deep_scrub_object
% Done:
0%
Source:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
v19.3.0-834-gdf1ae4de15
Released In:
v20.2.0~3278
Upkeep Timestamp:
2025-11-01T01:19:37+00:00
Description
sjust-2024-03-19_02:36:51-crimson-rados-wip-sjust-crimson-testing-2024-03-18-distro-default-smithi/7610285/
DEBUG 2024-03-19 03:41:57,451 [shard 0:main] bluestore - _add level 1 near 0 on buffer(0x60e00015c7e8 space 0x6110000b9e10 0x0~1000 clean) which has cache_private 0
DEBUG 2024-03-19 03:41:57,451 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-0) _generate_read_result_bl blob Blob(0x61100017aec0 blob([0x63000~1000] clen 0xf000 -> 0xb51 compressed+csum crc32c/0x1000/4) use_tracker(0xf000 0xf000) (shared_blob=NULL)) need {<0x0, 0xf000> : [0x1000:0~f000]}
DEBUG 2024-03-19 03:41:57,451 [shard 0:main] bluestore - bluestore.BufferSpace(0x61100017aed0 in 0x612000042640) _discard 0x0~f000
DEBUG 2024-03-19 03:41:57,451 [shard 0:main] bluestore - _add level 1 near 0 on buffer(0x60e0001a9708 space 0x61100017aed0 0x0~f000 clean) which has cache_private 0
DEBUG 2024-03-19 03:41:57,451 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-0) _generate_read_result_bl blob Blob(0x61100017b140 blob([0x64000~1000] clen 0xf000 -> 0xb72 compressed+csum crc32c/0x1000/4) use_tracker(0xf000 0xf000) (shared_blob=NULL)) need {<0x0, 0xf000> : [0x11000:0~f000]}
DEBUG 2024-03-19 03:41:57,451 [shard 0:main] bluestore - bluestore.BufferSpace(0x61100017b150 in 0x612000042640) _discard 0x0~f000
DEBUG 2024-03-19 03:41:57,452 [shard 0:main] bluestore - _add level 1 near 0 on buffer(0x60e00015b828 space 0x61100017b150 0x0~f000 clean) which has cache_private 0
DEBUG 2024-03-19 03:41:57,452 [shard 0:main] bluestore - maybe_unpin 0x625000028100 #1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head# touched
=================================================================
==29950==ERROR: AddressSanitizer: stack-use-after-return on address 0x7ff1a1f36d60 at pc 0x000004cb17a2 bp 0x7ffc7868eda0 sp 0x7ffc7868ed90
READ of size 4 at 0x7ff1a1f36d60 thread T0
Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb9853e9 0xb857eb0 0xb85942c 0xb859642 0xb859798 0xb859c61 0x54daf 0x131599 0x1331e6 0x133eba 0x11f834 0x11f955 0x10b7d9 0x10dfec 0x106210 0x1067ad 0x35190 0xe975f 0xe8c91 0xe9f5a 0x4cb17a1 0x4c803f3 0x4c80bd3 0x4c80db9 0x4c865ae 0x4c86882 0x4c86937 0x433afa8 0x433b220 0x433f199 0x43a5fdd 0xb86ab35 0xb884dcf 0xb925dcd 0xb92741b 0xb603b83 0xb6044ff 0x367c57a 0x3feaf
0x3ff5f 0x3468434
kernel callstack:
Reactor stalled for 122 ms on shard 0. Backtrace: 0x6bddd 0xb9853e9 0xb857eb0 0xb85942c 0xb859642 0xb859798 0xb859c61 0x54daf 0x12c797 0x131145 0x1331e6 0x133eba 0x11f834 0x11f955 0x10b7d9 0x10dfec 0x106210 0x1067ad 0x35190 0xe975f 0xe8c91 0xe9f5a 0x4cb17a1 0x4c803f3 0x4c80bd3 0x4c80db9 0x4c865ae 0x4c86882 0x4c86937 0x433afa8 0x433b220 0x433f199 0x43a5fdd 0xb86ab35 0xb884dcf 0xb925dcd 0xb92741b 0xb603b83 0xb6044ff 0x367c57
a 0x3feaf 0x3ff5f 0x3468434
kernel callstack:
Reactor stalled for 230 ms on shard 0. Backtrace: 0x6bddd 0xb9853e9 0xb857eb0 0xb85942c 0xb859642 0xb859798 0xb859c61 0x54daf 0x11994f 0x11ee25 0x1337f0 0x133c6d 0x1962d3 0xb6365d8 0x133f14 0x11f834 0x11f955 0x10b7d9 0x10dfec 0x106210 0x1067ad 0x35190 0xe975f 0xe8c91 0xe9f5a 0x4cb17a1 0x4c803f3 0x4c80bd3 0x4c80db9 0x4c865ae 0x4c86882 0x4c86937 0x433afa8 0x433b220 0x433f199 0x43a5fdd 0xb86ab35 0xb884dcf 0xb925dcd 0xb92741b
0xb603b83 0xb6044ff 0x367c57a 0x3feaf 0x3ff5f 0x3468434
kernel callstack:
#0 0x4cb17a1 in ceph::buffer::hash::update(ceph::buffer::v15_2_0::list const&) (/usr/bin/ceph-osd+0x4cb17a1) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#1 0x4c803f3 in auto crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}::operator()<ceph::buffer::v15_2_0::list>(ceph::buffer::v15_2_0::list) const (/usr/bin/ceph-osd+0x4c803f3) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#2 0x4c80bd3 in seastar::future<void> seastar::futurize<void>::invoke<crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, ceph::buffer::v15_2_0::list>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&, ceph::buffer::v15_2_0::list&&) (/usr/bin/ceph-osd+0x4c80bd3) (B
uildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#3 0x4c80db9 in crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::futurize<void>::invoke<crimson:
:osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, ceph::buffer::v15_2_0::list>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&, ceph::buffer::v15_2_0::list&&) (/usr/bin/ceph-osd+0x4c80db9) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#4 0x4c865ae in auto crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<ceph::buffer::v15_2_0::list> >::safe_then<crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, crimson::errorator<crimson
::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, c
rimson::ec<5> > >::pass_further&&)::{lambda(auto:1&&)#1}::operator()<seastar::future<ceph::buffer::v15_2_0::list> >(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&) (/usr/bin/ceph-osd+0x4c865ae) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#5 0x4c86882 in seastar::future<ceph::buffer::v15_2_0::list>::then_wrapped_maybe_erase<false, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> >, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrap
per<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<ceph::buffer::v15_2_0::list> >::safe_then<crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pas
s_further>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further&&)::{lambda(auto:1&&)#1}>(crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::e
c<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<ceph::buffer::v15_2_0::list> >::safe_then<crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_cod
e const&, crimson::ec<5> > >::pass_further>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further&&)::{lambda(auto:1&&)#1}&&)::{lambda(seastar::future<ceph::buffer::v15_2_0:
:list>&&)#1}::operator()(seastar::future<ceph::buffer::v15_2_0::list>&&) (/usr/bin/ceph-osd+0x4c86882) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#6 0x4c86937 in seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>::direct_vtable_for<seastar::future<ceph::buffer::v15_2_0::list>::then_wrapped_maybe_erase<false, crimson::errorato
r<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> >, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<ceph::buffer::v15_2_0::lis
t> >::safe_then<crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator(
)()::{lambda(auto:1)#1}&&, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further&&)::{lambda(auto:1&&)#1}>(crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_
marker<ceph::buffer::v15_2_0::list> >::safe_then<crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further>(crimson::osd::ScrubScan::deep_scan_object(crimson::osd::PG&, ghobject_t
const&)::{lambda()#1}::operator()()::{lambda(auto:1)#1}&&, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::pass_further&&)::{lambda(auto:1&&)#1}&&)::{lambda(seastar::future<ceph::buffer::v15_2_0::list>&&)#1}>::call(seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code c
onst&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)> const*, seastar::future<ceph::buffer::v15_2_0::list>&&) (/usr/bin/ceph-osd+0x4c86937) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#7 0x433afa8 in seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>::operator()(seastar::future<ceph::buffer::v15_2_0::list>&&) const (/usr/bin/ceph-osd+0x433afa8) (BuildId: b4225d19
d3f8d86df7d2e881368d445375bddbe5)
#8 0x433b220 in seastar::future<ceph::buffer::v15_2_0::list>::then_wrapped_nrvo<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> >, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::
unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)> >(seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ce
ph::buffer::v15_2_0::list>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&, seastar::future_state<ceph::buffer::v1
5_2_0::list>&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&, seastar::future_state<ceph::buffer::v15_2_0::list>
&&) const::{lambda()#1}::operator()() const (/usr/bin/ceph-osd+0x433b220) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#9 0x433f199 in void seastar::futurize<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > >::satisfy_with_result_of<seastar::internal::promise_base_with_type<void>, seastar::future<ceph::buffer::v15_2_0::list>::then_wrapped_nrvo<crimson::errorator<crimson::unth
rowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> >, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (se
astar::future<ceph::buffer::v15_2_0::list>&&)> >(seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_f
unction<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&, seastar::future_state<ceph::buffer::v15_2_0::list>&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crim
son::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&, seastar::future_state<ceph::buffer::v15_2_0::list>&&) const::{lambda()#1}>(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::e
rrorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&&) (/usr/bin/ceph-osd+0x433f199) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
#10 0x43a5fdd in seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>, seastar::future<ceph::buffer::v15_2_0::li
st>::then_wrapped_nrvo<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> >, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> >
>::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)> >(seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&&)::{lambda(seastar::internal:
:promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<5> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<ceph::buffer::v15_2_0::list>&&)>&, seastar::future_state<ceph::buffer::v15_2_0::list>&&)#1}, ceph::buffer::v15_2_0::list>::run_and_dis
pose() (/usr/bin/ceph-osd+0x43a5fdd) (BuildId: b4225d19d3f8d86df7d2e881368d445375bddbe5)
Updated by Samuel Just about 2 years ago
The instance above included the gcc-13 PR -- https://github.com/ceph/ceph/pull/55886. I can reproduce this locally with gcc-13, including on main and on sha1's with clean-ish recent pulpito runs, which suggests that this is somehow related to gcc-13. Going to check 11.
Updated by Samuel Just about 2 years ago
Not a gcc-13 bug -- seastar::repeat may move the passed action after the first invocation so the progress value may move. Simple fix.
Updated by Samuel Just about 2 years ago
- Status changed from New to Fix Under Review
Updated by Matan Breizman almost 2 years ago
- Status changed from Fix Under Review to Resolved
- Pull request ID set to 56375
Updated by Upkeep Bot 8 months ago
- Merge Commit set to df1ae4de15db3746ff6e055ed64b64c33b26fff9
- Fixed In set to v19.3.0-834-gdf1ae4de15d
- Upkeep Timestamp set to 2025-07-11T14:14:20+00:00
Updated by Upkeep Bot 8 months ago
- Fixed In changed from v19.3.0-834-gdf1ae4de15d to v19.3.0-834-gdf1ae4de15
- Upkeep Timestamp changed from 2025-07-11T14:14:20+00:00 to 2025-07-14T23:10:10+00:00
Updated by Upkeep Bot 5 months ago
- Released In set to v20.2.0~3278
- Upkeep Timestamp changed from 2025-07-14T23:10:10+00:00 to 2025-11-01T01:19:37+00:00
Actions