Parts metadata cache and failing mutation #39197
Description
BC check in Stress Tests sometimes fail with
2022.07.13 10:26:18.097238 [ 1051250 ] {} <Error> MutateFromLogEntryTask: virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 47. DB::Exception: Missing columns: 'v3' while processing query: 'v3, k, v1, v2, p', required columns: 'v3' 'k' 'v1' 'v2' 'p', maybe you meant: ['v1','k','v1','v2','p']. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):
2022.07.13 10:26:18.098966 [ 1051240 ] {} <Error> MutateFromLogEntryTask: virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 47. DB::Exception: Missing columns: 'v3' while processing query: 'v3, k, v1, v2, p', required columns: 'v3' 'k' 'v1' 'v2' 'p', maybe you meant: ['v1','k','v1','v2','p']. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):
2022.07.13 10:26:18.101742 [ 1051251 ] {} <Error> MutateFromLogEntryTask: virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 47. DB::Exception: Missing columns: 'v3' while processing query: 'v3, k, v1, v2, p', required columns: 'v3' 'k' 'v1' 'v2' 'p', maybe you meant: ['v1','k','v1','v2','p']. (UNKNOWN_IDENTIFIER), Stack trace (when copying this message, always include the lines below):
i.e. some mutation cannot be executed (slightly related to #39174). This mutation was started by the test 01233_check_table_with_metadata_cache. Logs:
by table id: https://pastila.nl/?024eb85d/65a0e50dba5aafb52d0ecec2cf2d5737
by query ids: https://pastila.nl/?002a7f2e/7505e0215864bbb1ea3da78c4a0151db
The second mutation (MODIFY TTL + implicit MATERIALIZE TTL) is trying to read from column that was dropped by the first mutation (DROP COLUMN v3). The error persists after server restart, so I suppose metadata was corrupted. Probably not metadata itself, but metadata cache.
cc: @taiyang-li, @CurtizJ