Skip to content

upgrades: report progress during 25.1 upgrade jobs backfill#141833

Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom
dt:progress
Feb 21, 2025
Merged

upgrades: report progress during 25.1 upgrade jobs backfill#141833
craig[bot] merged 2 commits intocockroachdb:masterfrom
dt:progress

Conversation

@dt
Copy link
Copy Markdown
Contributor

@dt dt commented Feb 21, 2025

Release note: none.
Epic: none.

dt added 2 commits February 21, 2025 13:22
When upgrades are running in real, non-test setups, they run in jobs. This
change passes the job ID, when it exists, to the upgrade func in its deps
struct, though the field name reflects that it can be absent (as in tests).

This allows the upgrade func to elect to load and use/mutate the job if it
so chooses.

Release note: none.
Epic: none.
@dt dt requested a review from msbutler February 21, 2025 14:31
@dt dt requested review from a team as code owners February 21, 2025 14:31
@blathers-crl

This comment was marked as off-topic.

@cockroach-teamcity

This comment was marked as off-topic.

@dt
Copy link
Copy Markdown
Contributor Author

dt commented Feb 21, 2025

Examples:
Screenshot 2025-02-21 at 09 30 21

Logs:

upgrade/upgrades/v25_1_add_jobs_tables.go:206 ⋮ [T1,Vsystem,n1,job=MIGRATION id=1048830688883736577,upgrade=24.3-upgrading-to-25.1-step-018] 1147  backfilled new columns for 106 of 1047 jobs so far

@github-actions
Copy link
Copy Markdown
Contributor

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note Threshold
sec/op 9.410m ±3% 9.376m ±9% ~ p=0.912 n=10 3.0%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 10.14k ±0% 10.15k ±0% ~ p=0.957 n=10 2.0%
B/op 2.181Mi ±1% 2.181Mi ±0% ~ p=0.971 n=10 2.0%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/5a2139b/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/5a2139b19e818a29a21333550421ac8a8862c2ab/bin/pkg_sql_tests benchdiff/5a2139b/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/5a2139b/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/3690e14/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/3690e1479b9747dd361fe10cd9b1ba0adff91e89/bin/pkg_sql_tests benchdiff/3690e14/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/3690e14/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=3690e14 --new=5a2139b ./pkg/sql/tests
⚪ Sysbench [KV, 1node, local, oltp_read_only]
Metric Old Commit New Commit Delta Note Threshold
sec/op 667.0µ ±1% 666.7µ ±1% ~ p=0.796 n=10 2.0%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 439.0 ±0% 439.0 ±0% ~ p=1.000 n=10 1.5%
B/op 254.2Ki ±0% 254.2Ki ±0% ~ p=0.541 n=10 1.5%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/5a2139b/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/5a2139b19e818a29a21333550421ac8a8862c2ab/bin/pkg_sql_tests benchdiff/5a2139b/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/5a2139b/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/3690e14/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/3690e1479b9747dd361fe10cd9b1ba0adff91e89/bin/pkg_sql_tests benchdiff/3690e14/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/3690e14/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/1node_local/oltp_read_only$ --old=3690e14 --new=5a2139b ./pkg/sql/tests
⚪ Sysbench [KV, 1node, local, oltp_write_only]
Metric Old Commit New Commit Delta Note Threshold
sec/op 1.312m ±1% 1.310m ±2% ~ p=0.247 n=10 2.5%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 1.391k ±0% 1.391k ±0% ~ p=0.421 n=10 1.8%
B/op 289.7Ki ±0% 289.8Ki ±0% ~ p=0.912 n=10 1.8%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/5a2139b/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/5a2139b19e818a29a21333550421ac8a8862c2ab/bin/pkg_sql_tests benchdiff/5a2139b/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/5a2139b/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/3690e14/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/3690e1479b9747dd361fe10cd9b1ba0adff91e89/bin/pkg_sql_tests benchdiff/3690e14/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/3690e14/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/1node_local/oltp_write_only$ --old=3690e14 --new=5a2139b ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/5a2139b19e818a29a21333550421ac8a8862c2ab/13458923467-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/3690e1479b9747dd361fe10cd9b1ba0adff91e89/13458923467-1/\* old/
Legend
  • Neutral: No significant performance change.
  • 🟡 Warning: Slight degradation, likely due to variance, but still within thresholds.
  • 🔴 Regression: Likely performance regression, requiring investigation.
  • 🟢 Improvement: Possible performance gain.

No regressions detected!

built with commit: 5a2139b19e818a29a21333550421ac8a8862c2ab

@dt
Copy link
Copy Markdown
Contributor Author

dt commented Feb 21, 2025

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 21, 2025

@craig craig bot merged commit 5d1c0f8 into cockroachdb:master Feb 21, 2025
24 checks passed
@dt dt deleted the progress branch February 24, 2025 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants