Skip to content

expression: Fix different behaviors with MySQL when comparing datetime column with numeric constant | tidb-test=pr/2196#45945

Merged
ti-chi-bot[bot] merged 7 commits intopingcap:masterfrom
yibin87:fix_38361
Aug 14, 2023
Merged

expression: Fix different behaviors with MySQL when comparing datetime column with numeric constant | tidb-test=pr/2196#45945
ti-chi-bot[bot] merged 7 commits intopingcap:masterfrom
yibin87:fix_38361

Conversation

@yibin87
Copy link
Contributor

@yibin87 yibin87 commented Aug 9, 2023

What problem does this PR solve?

Issue Number: close #38361

Problem Summary: When compare datetime/timestamp column with numeric constant, try convert numeric constant to timestamp datetime, do nothing if conversion failed.

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: yibin <huyibin@pingcap.com>
@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 9, 2023
@tiprow
Copy link

tiprow bot commented Aug 9, 2023

Hi @yibin87. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: yibin <huyibin@pingcap.com>
Signed-off-by: yibin <huyibin@pingcap.com>
@codecov
Copy link

codecov bot commented Aug 10, 2023

Codecov Report

Merging #45945 (b25d708) into master (cdeac4c) will decrease coverage by 0.8829%.
Report is 42 commits behind head on master.
The diff coverage is 93.1818%.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #45945        +/-   ##
================================================
- Coverage   73.3497%   72.4669%   -0.8829%     
================================================
  Files          1277       1300        +23     
  Lines        393388     405127     +11739     
================================================
+ Hits         288549     293583      +5034     
- Misses        86440      92961      +6521     
- Partials      18399      18583       +184     
Flag Coverage Δ
integration 26.5537% <22.7272%> (?)
unit 73.3479% <93.1818%> (-0.0018%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 54.0444% <ø> (ø)
parser 85.0494% <ø> (+0.0129%) ⬆️
br 47.6015% <ø> (-4.4359%) ⬇️

Signed-off-by: yibin <huyibin@pingcap.com>
@yibin87
Copy link
Contributor Author

yibin87 commented Aug 10, 2023

/test unit-test

@tiprow
Copy link

tiprow bot commented Aug 10, 2023

@yibin87: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@yibin87 yibin87 changed the title expression: Fix different behaviors with MySQL when comparing datetime column with numeric constant expression: Fix different behaviors with MySQL when comparing datetime column with numeric constant | tidb-test=pr/2196 Aug 10, 2023
@yibin87
Copy link
Contributor Author

yibin87 commented Aug 10, 2023

/test mysql-test

@tiprow
Copy link

tiprow bot commented Aug 10, 2023

@yibin87: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/test mysql-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@yibin87
Copy link
Contributor Author

yibin87 commented Aug 10, 2023

/test unit-test

@tiprow
Copy link

tiprow bot commented Aug 10, 2023

@yibin87: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@yibin87
Copy link
Contributor Author

yibin87 commented Aug 10, 2023

/test unit-test

@tiprow
Copy link

tiprow bot commented Aug 10, 2023

@yibin87: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@yibin87
Copy link
Contributor Author

yibin87 commented Aug 10, 2023

/cc @windtalker @qw4990 @wshwsh12

@ti-chi-bot ti-chi-bot bot requested review from qw4990, windtalker and wshwsh12 August 10, 2023 09:57
tk.MustQuery("select a > 20230809 from t").Check(testkit.Rows("0"))
tk.MustQuery("select a = 20230809 from t").Check(testkit.Rows("1"))
tk.MustQuery("select a < 20230810 from t").Check(testkit.Rows("1"))
tk.MustQuery("select a < 20231310 from t").Check(testkit.Rows("0"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the case that numeric can't convert to datetime, so use real data type to compare?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some comments here?

}
sc := ctx.GetSessionVars().StmtCtx
var timestampDatum types.Datum
targetFieldType := types.NewFieldType(mysql.TypeTimestamp)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the target type is timestamp instead of datetime?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to MySQL document, "If one of the arguments is a TIMESTAMP or DATETIME column and the other argument is a constant, the constant is converted to a timestamp before the comparison is performed"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check that the MySQL seems convert to datetime instead, because it can handle numeric constant that is beyond timestamp's valid value range.

timestampDatum, err = dt.ConvertTo(sc, targetFieldType)
if err != nil {
return args
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe also need to return original args if timestampDatum is null but dt is not null

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, add it.

Signed-off-by: yibin <huyibin@pingcap.com>
@yibin87 yibin87 requested a review from windtalker August 11, 2023 05:14
Signed-off-by: yibin <huyibin@pingcap.com>
Copy link
Contributor

@windtalker windtalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Others LGTM

tk.MustQuery("select a > 20230809 from t").Check(testkit.Rows("0"))
tk.MustQuery("select a = 20230809 from t").Check(testkit.Rows("1"))
tk.MustQuery("select a < 20230810 from t").Check(testkit.Rows("1"))
tk.MustQuery("select a < 20231310 from t").Check(testkit.Rows("0"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some comments here?

var datetimeDatum types.Datum
targetFieldType := types.NewFieldType(mysql.TypeDatetime)
datetimeDatum, err = dt.ConvertTo(sc, targetFieldType)
if err != nil || datetimeDatum.IsNull() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need to check dt.IsNull() here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, added the comments and check.

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Aug 11, 2023
Signed-off-by: yibin <huyibin@pingcap.com>
@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 14, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: windtalker, wshwsh12

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [windtalker,wshwsh12]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Aug 14, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 14, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-08-11 08:45:44.469878488 +0000 UTC m=+275109.018894460: ☑️ agreed by windtalker.
  • 2023-08-14 08:14:44.65055675 +0000 UTC m=+532449.199572736: ☑️ agreed by wshwsh12.

@ti-chi-bot ti-chi-bot bot merged commit d5fc8bb into pingcap:master Aug 14, 2023
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-6.1: #46062.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-6.5: #46064.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 14, 2023
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-5.3: #46066.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-5.4: #46068.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 14, 2023
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@windtalker windtalker added the needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. label Aug 15, 2023
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.1: #46087.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong results and nonoptimal plan when datetime column comparing number constant

4 participants