Skip to content

Allow to attach a part/partition from a table with more granular partitioning. #13826

@filimonov

Description

@filimonov

if there are 2 table which differs only in the partitioning key and the part from source table matches single partition in target table - allow that (may be with setting like 'allow_attaching_compatible_partitioning_key_parts`).

create table source engine=MergeTree ORDER BY tuple() PARTITION BY toYYYYMMDD(timestamp) AS SELECT number, toDateTime('2010-01-01 00:00:00') as timestamp from numbers(100);

create table target engine=MergeTree ORDER BY tuple() PARTITION BY toYYYYMM(timestamp) AS SELECT number, toDateTime('2010-01-01 00:00:00') as timestamp from numbers(100);

ALTER TABLE target ATTACH PARTITION ID '20100101' FROM source;
Code: 36, e.displayText() = DB::Exception: Tables have different partition key (version 20.7.1.4186 (official build)) (from 127.0.0.1:39986) (in query: ALTER TABLE target ATTACH PARTITION ID '20100101' FROM source;), Stack trace (when copying this message, always include the lines below):

0. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x123b04c0 in /usr/lib/debug/usr/bin/clickhouse
1. /build/obj-x86_64-linux-gnu/../src/Common/Exception.cpp:37: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0xa2a02bd in /usr/lib/debug/usr/bin/clickhouse
2. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/string:2134: DB::MergeTreeData::checkStructureAndGetMergeTreeData(DB::IStorage&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) const (.cold) @ 0xf8a32f9 in /usr/lib/debug/usr/bin/clickhouse
3. /build/obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1146: DB::StorageMergeTree::replacePartitionFrom(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::IAST> const&, bool, DB::Context const&) @ 0xf6a7100 in /usr/lib/debug/usr/bin/clickhouse
4. /build/obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1057: DB::StorageMergeTree::alterPartition(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::vector<DB::PartitionCommand, std::__1::allocator<DB::PartitionCommand> > const&, DB::Context const&) @ 0xf6aa4bf in /usr/lib/debug/usr/bin/clickhouse
5. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:662: DB::InterpreterAlterQuery::execute() @ 0xf169433 in /usr/lib/debug/usr/bin/clickhouse
6. /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:386: DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) @ 0xf4d1bc2 in /usr/lib/debug/usr/bin/clickhouse
7. /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:655: DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) @ 0xf4d5405 in /usr/lib/debug/usr/bin/clickhouse
8. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:253: DB::TCPHandler::runImpl() @ 0xfb39ba3 in /usr/lib/debug/usr/bin/clickhouse
9. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1215: DB::TCPHandler::run() @ 0xfb3a99e in /usr/lib/debug/usr/bin/clickhouse
10. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x122ce2eb in /usr/lib/debug/usr/bin/clickhouse
11. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:856: Poco::Net::TCPServerDispatcher::run() @ 0x122ce77b in /usr/lib/debug/usr/bin/clickhouse
12. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59: Poco::PooledThread::run() @ 0x1244d2a6 in /usr/lib/debug/usr/bin/clickhouse
13. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223: Poco::ThreadImpl::runnableEntry(void*) @ 0x124486a0 in /usr/lib/debug/usr/bin/clickhouse
14. start_thread @ 0x9609 in /lib/x86_64-linux-gnu/libpthread-2.31.so
15. /build/glibc-YYA7BZ/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97: __clone @ 0x122103 in /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.31.so


Received exception from server (version 20.7.1):
Code: 36. DB::Exception: Received from localhost:9000. DB::Exception: Tables have different partition key. 

Internally what should happen:

  1. we copy (by hardlinks) the part (as now)
  2. we recalculate min/max for the column used in partitioning key if it ends up with different partitions - fail with exception, if it's a single partitions - write new minmax file (see IMergeTreeDataPart::MinMaxIndex)
  3. recalc checksum file
  4. do rest as usual

Metadata

Metadata

Assignees

Labels

featurest-acceptedThe issue is in our backlog, ready to take

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions