Changelog category (leave one):

• Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

This closes #48691.

Details:

Alexey Milovidov
@alexander Tokmakov, you should know about this: https://s3.amazonaws.com/clickhouse-test-reports/50533/009fe3d25e8a755e86d45084f2af57[…]ease__[2_4]/integration_run_test_merge_tree_s3_test_py_0.log
(this test fails very frequently)
PS. I checked the code DatabaseCatalog::cleanupStoreDirectoryTask and it looks dangerous - we are removing data when it is unexpected. Maybe better do nothing (when we found some garbage, we should not touch it, or move to a safe place, instead of removing).

Alexey Milovidov
@igor Nikonov, if you attempted to fix this test, you might want to know that it didn't help.

Alexey Milovidov
@alexander Tokmakov, what is the whole motivation of this test_store_cleanup_disk_s3 ?
It does manual intervention to filesystem - not a scenario to expect in production. Maybe remove it?

Alexey Milovidov
If you know the motivation for this test - provide it directly in comments.

Alexander Tokmakov
The motivation is: check that background cleanup of leftovers works as expected (it's not easy to create some leftovers in a natural way). I thought that @igor Nikonov is working on fixing the test (#48691 is assigned to him), but I will take a look by myself if he cannot do it
PS. I checked the code DatabaseCatalog::cleanupStoreDirectoryTask and it looks dangerous - we are removing data when it is unexpected. Maybe better do nothing (when we found some garbage, we should not touch it, or move to a safe place, instead of removing).

At first we set permissions to 000 and only then remove after one month. And in tests we remove it after 1 second, no issues found. So it's not that dangerous. Initially we used to do nothing, but Yandex had some complaints about garbage in store/ dir that was hard to remove manually
#48691 test_store_cleanup_disk_s3 is flaky
https://s3.amazonaws.com/clickhouse-test-reports/0/c3482a104f73f237450f7d7781871859472687d9/integration_tests__asan__[6/6].html
https://play.clickhouse.com/play?user=play#U0VMRUNUIHB1bGxfcmVxdWVzdF9udW1iZXIsIGNoZWNrX3N0YXJ0X3RpbWUsIGNoZWNrX25hbWUsIHRlc3RfbmFtZSwgdGVzdF9zdGF0dXMsIGNoZWNrX3N0YXR1cywgcmVwb3J0X3VybApGUk9NIGNoZWNrcwpXSEVSRSAxCiAgICAtLUFORCBwdWxsX3JlcXVlc3RfbnVtYmVyID0gMAogICAgQU5EIHRlc3Rfc3RhdHVzICE9ICdTS0lQUEVEJwogICAgQU5EIHRlc3Rfc3RhdHVzICE9ICdPSycKICAgIEFORCBjaGVja19zdGF0dXMgIT0gJ3N1Y2Nlc3MnCiAgICBBTkQgdGVzdF9uYW1lIGxpa2UgJyV0ZXN0X3N0b3JlX2NsZWFudXBfZGlza19zMyUnCk9SREVSIEJZIGNoZWNrX3N0YXJ0X3RpbWUgZGVzYywgY2hlY2tfbmFtZSwgdGVzdF9uYW1l

Alexey Milovidov
I see we failed to fix it in two months, do you mind if I'll remove this test?

TLDR: the feature should not have existed in the first place.