Skip to content

session: refine error message desensitization#19409

Merged
crazycs520 merged 34 commits intopingcap:masterfrom
crazycs520:fix-err-desentilize
Sep 15, 2020
Merged

session: refine error message desensitization#19409
crazycs520 merged 34 commits intopingcap:masterfrom
crazycs520:fix-err-desentilize

Conversation

@crazycs520
Copy link
Contributor

@crazycs520 crazycs520 commented Aug 24, 2020

Signed-off-by: crazycs520 crazycs520@gmail.com

What problem does this PR solve?

Close #18566 (comment)

  • Make Duplicate entry error message desensitization.

Below is the test SQL:

create table t (id int auto_increment, b int,  unique index (id));
insert into t values  (1,1),(1,1);
begin;
insert into t values  (1,1);
insert into t values  (1,1);
  • change variable name from tidb_log_desensitization to tidb_redact_log .

related log:

[2020/08/24 20:26:01.567 +08:00] [WARN] [session.go:476] ["can not retry txn"] [conn=1] [label=general] [error="[kv:1062]Duplicate entry '?' for key '?'"] [IsBatchInsert=false] [IsPessimistic=false] [InRestrictedSQL=false] [tidb_retry_limit=10] [tidb_disable_txn_auto_retry=true]
[2020/08/24 20:26:01.567 +08:00] [WARN] [session.go:492] ["commit failed"] [conn=1] ["finished txn"="Txn{state=invalid}"] [error="[kv:1062]Duplicate entry '?' for key '?'"]
[2020/08/24 20:26:01.567 +08:00] [ERROR] [conn.go:744] ["command dispatched failed"] [conn=1] [connInfo="id:1, addr:127.0.0.1:60651 status:10, collation:utf8_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="insert into t values ( ... )"] [txn_mode=OPTIMISTIC] [err="[kv:1062]Duplicate entry '?' for key '?'"]
[2020/08/24 20:26:01.567 +08:00] [INFO] [2pc.go:717] ["2PC clean up done"] [conn=1] [txnStartTS=418977405093019648]
[2020/08/24 20:26:01.663 +08:00] [ERROR] [conn.go:744] ["command dispatched failed"] [conn=1] [connInfo="id:1, addr:127.0.0.1:60651 status:11, collation:utf8_general_ci, user:root"] [command=Query] [status="inTxn:1, autocommit:1"] [sql="insert into t values ( ... )"] [txn_mode=OPTIMISTIC] [err="[kv:1062]Duplicate entry '?' for key '?'"]

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
    • Breaking backward compatibility: change variable name from tidb_log_desensitization to tidb_redact_log .

Release note

  • Refine error message desensitization.

Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520 crazycs520 added the security Everything related with security label Aug 24, 2020
@crazycs520
Copy link
Contributor Author

@bb7133 @jackysp PTAL

Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
Copy link
Contributor Author

/rebuild

Copy link
Contributor

@jackysp jackysp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 26, 2020
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520 crazycs520 requested a review from a team as a code owner September 4, 2020 13:24
@crazycs520 crazycs520 requested review from XuHuaiyu and removed request for a team September 4, 2020 13:24
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
Copy link
Contributor Author

/run-all-tests

@XuHuaiyu XuHuaiyu removed their request for review September 14, 2020 08:10
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Sep 15, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Sep 15, 2020
@SunRunAway
Copy link
Contributor

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 15, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@crazycs520 merge failed.

@crazycs520
Copy link
Contributor Author

/run-integration-common-test

@crazycs520 crazycs520 merged commit 70a567e into pingcap:master Sep 15, 2020
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Sep 15, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #20004

crazycs520 added a commit to ti-srebot/tidb that referenced this pull request Sep 17, 2020
ti-srebot added a commit that referenced this pull request Sep 17, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: crazycs520 <crazycs520@gmail.com>
SunRunAway added a commit to SunRunAway/docs-cn that referenced this pull request Dec 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/config security Everything related with security sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support error/info log desensitization

5 participants