Skip to content

Mask some information in logs#42484

Merged
vitlibar merged 15 commits intoClickHouse:masterfrom
vitlibar:mask-sensitive-info-in-logs
Nov 4, 2022
Merged

Mask some information in logs#42484
vitlibar merged 15 commits intoClickHouse:masterfrom
vitlibar:mask-sensitive-info-in-logs

Conversation

@vitlibar
Copy link
Copy Markdown
Member

@vitlibar vitlibar commented Oct 19, 2022 •
edited
Loading

Changelog category:

  • Improvement

Changelog entry:

Mask passwords and secret keys both in system.query_log and /var/log/clickhouse-server/*.log and also in error messages.

For example,

CREATE TABLE table2 (`x` int) ENGINE = MongoDB('mongo1:27017', 'db', 'col', 'mongo_user', 'mongo_password')

will be written in logs from now on as

CREATE TABLE table2 (`x` int) ENGINE = MongoDB('mongo1:27017', 'db', 'col', 'mongo_user', '[HIDDEN]')

@vitlibar vitlibar changed the title Mask sensitive info in logs Mask some information in logs Oct 19, 2022
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-not-for-changelog This PR should not be mentioned in the changelog label Oct 19, 2022
@vitlibar vitlibar force-pushed the mask-sensitive-info-in-logs branch 5 times, most recently from 1f23a68 to ff5de4e Compare October 20, 2022 17:03
@SmitaRKulkarni SmitaRKulkarni self-assigned this Oct 21, 2022
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 24, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 24, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 24, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 24, 2022
View reviewed changes
@vitlibar vitlibar force-pushed the mask-sensitive-info-in-logs branch 4 times, most recently from adc2de6 to b2076d1 Compare October 25, 2022 06:16
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 25, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 25, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 25, 2022
View reviewed changes
@vitlibar vitlibar force-pushed the mask-sensitive-info-in-logs branch 2 times, most recently from 8e44ac6 to 5e87f1d Compare October 26, 2022 12:01
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 27, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 27, 2022
View reviewed changes
SmitaRKulkarni
SmitaRKulkarni reviewed Oct 27, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@SmitaRKulkarni SmitaRKulkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest all LGTM

@vitlibar vitlibar force-pushed the mask-sensitive-info-in-logs branch from 25cb5ba to cc2505d Compare October 28, 2022 06:45
@vitlibar vitlibar force-pushed the mask-sensitive-info-in-logs branch from cc2505d to d1c69a0 Compare October 31, 2022 16:08
@nikitamikhaylov
Copy link
Copy Markdown
Member

nikitamikhaylov commented Nov 1, 2022 •
edited
Loading

What about queries for creating dictionaries? Will they also be masked?

CREATE DICTIONARY test_dictionary
(
    `id` UInt64,
    `value` String
)
PRIMARY KEY id
SOURCE(CLICKHOUSE(TABLE 'test_table' PASSWORD 'password'))
LAYOUT(DIRECT())

@vitlibar
Copy link
Copy Markdown
Member Author

vitlibar commented Nov 1, 2022 •
edited
Loading

What about queries for creating dictionaries? Will they also be masked?

CREATE DICTIONARY test_dictionary
(
    `id` UInt64,
    `value` String
)
PRIMARY KEY id
SOURCE(CLICKHOUSE(TABLE 'test_table' PASSWORD 'password'))
LAYOUT(DIRECT())

yes

@vitlibar vitlibar force-pushed the mask-sensitive-info-in-logs branch from 3719447 to 085fb80 Compare November 1, 2022 16:20
@alexey-milovidov
Copy link
Copy Markdown
Member

alexey-milovidov commented Nov 2, 2022

Not for changelog

Why not? We can even highlight this change on the release webinar.

@vitlibar vitlibar merged commit 52b1f4a into ClickHouse:master Nov 4, 2022
@vitlibar vitlibar deleted the mask-sensitive-info-in-logs branch November 4, 2022 13:09
@vitlibar vitlibar mentioned this pull request Nov 7, 2022
Merged
@vitlibar vitlibar mentioned this pull request Nov 14, 2022
Merged
@Ryado Ryado mentioned this pull request Apr 19, 2023
Closed
@nikitamikhaylov nikitamikhaylov mentioned this pull request Jan 23, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SmitaRKulkarni SmitaRKulkarni SmitaRKulkarni approved these changes

Assignees

@SmitaRKulkarni SmitaRKulkarni

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@vitlibar @nikitamikhaylov @alexey-milovidov @SmitaRKulkarni @robot-ch-test-poll2