Secure inter-cluster query execution (with initial_user as current query user) [v3] by azat · Pull Request #13156 · ClickHouse/ClickHouse

azat · 2020-07-30T23:16:54Z

Changelog category (leave one):

New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Secure inter-cluster query execution (with initial_user as current query user)

Detailed description / Documentation draft:

Implemented:

Add inter-server cluster secret (to verify the query and all related)
Use initial_user as current query user (to apply correct *_for_user settings)

Usage example:

set remote_servers.$CLUSTER.secret to use initial_user as current query user (if omitted/empty then works as before)

v1: #11391
v2: #11498
Fixes: #6843
Fixes: #9751
Cc: @filimonov
Cc: @vitlibar

P.S. Not sure that this should be New Feature, marked it so to highlight this

Details

HEAD archive:

57eef2b8a1e85d40611950755c2294145b73ef1b # BEGINNING
f74295c61f6bf1577e49724974da34ed2776fa9a
dfb70cf7d41de725bbc71c8da86d648cb27f8464 (inter-server user marker)

HEAD pre-merge:

c1a61eed4362cfb1745a012a37a96358997356f4

azat · 2020-08-01T08:47:51Z

Integration tests (asan) — fail: 12, passed: 761, error: 0
Integration tests (release) — fail: 12, passed: 760, error: 0

test_cluster_copier - does not looks related

src/Client/Connection.cpp

azat · 2020-08-04T06:40:23Z

@vitlibar can you take a look please?

src/Server/TCPHandler.cpp

src/Interpreters/Context.cpp

src/Server/TCPHandler.cpp

azat · 2020-08-11T21:33:32Z

AST fuzzer — Fuzzer failed (134). See the logs

SELECT arrayJoin([[], []]), * FROM numbers(1) GROUP BY number WITH TOTALS

Fix - #13625

azat · 2020-08-12T22:15:49Z

Functional stateless tests (release, DatabaseAtomic) — fail: 1, passed: 2151, skipped: 20

00633_materialized_view_and_too_many_parts_zookeeper does not looks related

src/Server/TCPHandler.cpp

azat · 2020-08-13T22:20:16Z

AST fuzzer — Logical error: 'Bad cast from type DB::ColumnVector to DB::ColumnString'.

Can't reproduce the problem (tried to locate server log with brute force, no luck)

azat · 2020-08-15T17:11:17Z

Functional stateless tests (debug) — fail: 1, passed: 2159, skipped: 13
Functional stateless tests (memory) — fail: 1, passed: 2164, skipped: 8
Functional stateless tests (thread) — fail: 1, passed: 2162, skipped: 10

Unrelated (fixed in #13746)

Integration tests (release) — fail: 1, passed: 782, error: 0

test_storage_rabbitmq - know flacky test

AST fuzzer — Fuzzer failed (134). See the logs

Known problem with arrayJoin GROUP BY WITH TOTALS (#13625)

src/Client/Connection.cpp

azat · 2020-09-16T17:06:48Z

@akuzm looks like performance tests hanged, can you take a look please?

akuzm · 2020-09-17T09:17:34Z

@akuzm looks like performance tests hanged, can you take a look please?

They are waiting in queue (even now, 16 hour later...). I increased the priority.

azat marked this pull request as draft July 30, 2020 23:17

robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Jul 30, 2020

azat force-pushed the cluster-secure branch 2 times, most recently from 70ba6c6 to c75f974 Compare July 31, 2020 23:43

azat marked this pull request as ready for review July 31, 2020 23:43

azat force-pushed the cluster-secure branch 2 times, most recently from e92d166 to 3e06b47 Compare August 2, 2020 08:28

filimonov reviewed Aug 3, 2020

View reviewed changes

src/Client/Connection.cpp Outdated Show resolved Hide resolved

vitlibar self-assigned this Aug 4, 2020

vitlibar reviewed Aug 4, 2020

View reviewed changes