Skip to content

sql: increase tenant testing coverage#141604

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
shubhamdhama:sql-tenant-testing-part-2
Feb 21, 2025
Merged

sql: increase tenant testing coverage#141604
craig[bot] merged 1 commit intocockroachdb:masterfrom
shubhamdhama:sql-tenant-testing-part-2

Conversation

@shubhamdhama
Copy link
Copy Markdown
Contributor

@shubhamdhama shubhamdhama commented Feb 18, 2025

This PR continues the work from #140447, replacing occurrences of
createTestServerParams with createTestServerParamsAllowTenants
to enable tenant testing in these tests.

Informs: #140446
Epic: CRDB-38970
Release note: None

@shubhamdhama shubhamdhama requested review from a team as code owners February 18, 2025 05:56
@shubhamdhama shubhamdhama requested a review from a team February 18, 2025 05:56
@shubhamdhama shubhamdhama requested review from a team as code owners February 18, 2025 05:56
@shubhamdhama shubhamdhama requested review from DarrylWong, golgeek, kev-cao, kyle-a-wong, mgartner and wenyihu6 and removed request for a team February 18, 2025 05:56
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the pkg/sql changes lgtm, but some of these tests are owned by the SQL Queries and DR teams, so they should take a look.

Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2nd commit :lgtm:

Reviewed 15 of 15 files at r1, 16 of 16 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @DarrylWong, @golgeek, @kev-cao, @kyle-a-wong, @mgartner, and @wenyihu6)

(Stacked on cockroachdb#141490.)

This PR continues the work from cockroachdb#140447, replacing occurrences of
`createTestServerParams` with `createTestServerParamsAllowTenants`
to enable tenant testing in these tests.

Informs: cockroachdb#140446
Epic: CRDB-38970
Release note: None
@shubhamdhama shubhamdhama force-pushed the sql-tenant-testing-part-2 branch from bf68b79 to 6f2435c Compare February 21, 2025 12:34
@github-actions
Copy link
Copy Markdown
Contributor

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note Threshold
sec/op 9.640m ±2% 9.690m ±3% ~ p=0.684 n=10 3.0%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 10.16k ±1% 10.15k ±0% ~ p=0.470 n=10 2.0%
B/op 2.182Mi ±0% 2.181Mi ±0% ~ p=0.971 n=10 2.0%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/6f2435c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/bin/pkg_sql_tests benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/fbcabe0/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/fbcabe029eb2e1701710d08cebbca6d15089f911/bin/pkg_sql_tests benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=fbcabe0 --new=6f2435c ./pkg/sql/tests
⚪ Sysbench [KV, 1node, local, oltp_read_only]
Metric Old Commit New Commit Delta Note Threshold
sec/op 672.5µ ±1% 679.5µ ±2% ~ p=0.315 n=10 2.0%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 439.0 ±0% 439.0 ±0% ~ p=1.000 n=10 1.5%
B/op 254.2Ki ±0% 254.2Ki ±0% ~ p=0.210 n=10 1.5%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/6f2435c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/bin/pkg_sql_tests benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/fbcabe0/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/fbcabe029eb2e1701710d08cebbca6d15089f911/bin/pkg_sql_tests benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/1node_local/oltp_read_only$ --old=fbcabe0 --new=6f2435c ./pkg/sql/tests
⚪ Sysbench [KV, 1node, local, oltp_write_only]
Metric Old Commit New Commit Delta Note Threshold
sec/op 1.338m ±1% 1.341m ±1% ~ p=0.684 n=10 2.5%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 1.394k ±0% 1.394k ±0% ~ p=0.469 n=10 1.8%
B/op 290.3Ki ±0% 290.1Ki ±0% ~ p=0.436 n=10 1.8%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/6f2435c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/bin/pkg_sql_tests benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/fbcabe0/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/fbcabe029eb2e1701710d08cebbca6d15089f911/bin/pkg_sql_tests benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/1node_local/oltp_write_only$ --old=fbcabe0 --new=6f2435c ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/13456892060-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/fbcabe029eb2e1701710d08cebbca6d15089f911/13456892060-1/\* old/
Legend
  • Neutral: No significant performance change.
  • 🟡 Warning: Slight degradation, likely due to variance, but still within thresholds.
  • 🔴 Regression: Likely performance regression, requiring investigation.
  • 🟢 Improvement: Possible performance gain.

No regressions detected!

built with commit: 6f2435c712e4ca5c7b72293bfb605a1f70bb998d

@shubhamdhama
Copy link
Copy Markdown
Contributor Author

shubhamdhama commented Feb 21, 2025

After removing the stacked commit, it seems there are no remaining changes that would affect the DR team. So, we should be good to merge. Thanks for the reviews!

bors r =rafiss,yuzefovich

@shubhamdhama
Copy link
Copy Markdown
Contributor Author

Oops!

bors r=rafiss,yuzefovich

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 21, 2025

@craig craig bot merged commit 5d1c0f8 into cockroachdb:master Feb 21, 2025
24 checks passed
@shubhamdhama shubhamdhama deleted the sql-tenant-testing-part-2 branch March 1, 2025 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants