Revisions and Binaries

Master: cc99062
./dev test-binaries ./pkg/sql/... --output=test_binaries.tar.gz
22.2.3: 81a114c
./dev test-binaries ./pkg/sql/... --output=test_binaries_compare.tar.gz

Execution

Cluster: ./bin/roachprod create herko-bench -n 16 --lifetime "24h" --clouds gce --gce-machine-type "n1-standard-8" --gce-zones="us-east4-c" --os-volume-size=200
Timeout: 20m
Iterations: 10
Benchtime: 1s
Random Seed: 1

./dev build roachprod-microbench
./bin/roachprod-microbench ./artifacts/log -cluster herko-bench -compare-binaries ./bin/test_binaries_compare.tar.gz -shell="export COCKROACH_RANDOM_SEED=1" -iterations=10 -timeout=20m -- -test.benchtime=1s

Analytics

Iterations per microbenchmark, duration of each iteration: https://docs.google.com/spreadsheets/d/1IJ9-lbMo2mJ_O-NoQU77iRYvuElRwjaIbajPD5BNUhU/edit?usp=sharing

Timeouts:

Timeouts are on a per microbenchmark function level.
Either the whole microbenchmark timed out, or some of the permutations.

BenchmarkDatabaseBackup
BenchmarkDatabaseRestore
BenchmarkIntentRangeResolution
BenchmarkMVCCGarbageCollect
BenchmarkMVCCGet_Pebble
BenchmarkMVCCReverseScan_Pebble
BenchmarkMVCCScanGarbage_Pebble
BenchmarkMVCCScan_Pebble

Open Issues:

Failures:

• bench: BenchmarkOrderBy fails with liveness session expired #96336
• storage: BenchmarkMVCCExportToSST fails on numRangeKeys #96209
• kv/kvserver: BenchmarkMVCCGCWithForegroundTraffic fails with expected nil #96266
• ccl/workloadccl: BenchmarkImportFixture fails with memory budget exceeded #96013

Sheets

Generated sheet for pkg/sql/...
https://docs.google.com/spreadsheets/d/1uDYotBARIqdL5nSquMoF7Fy3qqoMfNigwHAcB8NPR7M/edit
Generated sheet for pkg/storage/...
https://docs.google.com/spreadsheets/d/1kagTm9WXv9CSn78nDg3iD0bsnFXhc-r4a0L29hP_-No/edit
Generated sheet for pkg/col/...
https://docs.google.com/spreadsheets/d/1VZPc_Ft9RKK60OcjO_x2zKv9LttZ--U3SVsxeEwFcik/edit
Generated sheet for pkg/workload/...
https://docs.google.com/spreadsheets/d/1jgaXqS_3DikzbTxP5WsJ_7gnDrX6jzTNr0ufmCrGK_8/edit
Generated sheet for pkg/ts/...
https://docs.google.com/spreadsheets/d/1r2IXm1gZviRVPIwZZHGQCZUTwY_N12WZrdX1TDpx4yI/edit
Generated sheet for pkg/roachpb/...
https://docs.google.com/spreadsheets/d/1Ejd09DpvXszNYHYf82Sg857tZscJURaScipU_UtXO6E/edit
Generated sheet for pkg/server/...
https://docs.google.com/spreadsheets/d/1phImav7bvnQEzd8okFjDRjoxPMlptevEs7M7a9kgtIQ/edit
Generated sheet for pkg/bench/...
https://docs.google.com/spreadsheets/d/1l3BnrnfTYL6BudvLfksRzccoQwYX3ShwaV36-OU8Hhk/edit
Generated sheet for pkg/gossip/...
https://docs.google.com/spreadsheets/d/17ls1Zk23E3_27aP4supiXbph-kOsFYe0DsMpNqHPRg4/edit
Generated sheet for pkg/geo/...
https://docs.google.com/spreadsheets/d/1G0rfnnBNPZYjgpiRMokNB4diXo7rrV2VkspqXwZclOQ/edit
Generated sheet for pkg/blobs/...
https://docs.google.com/spreadsheets/d/1ksRJCTSFVBm0Y-whw0_1JXBf_GHWDTqdMISuOHpzG34/edit
Generated sheet for pkg/spanconfig/...
https://docs.google.com/spreadsheets/d/18oTvIY28XEZAEtajTxETM8xrj_cP2g5-j6_cB6dbrjs/edit
Generated sheet for pkg/ccl/...
https://docs.google.com/spreadsheets/d/1HxTZbtEMscgzt_4f-z1D2KB6HdvXw-qGaZF4zkXV1ik/edit
Generated sheet for pkg/kv/...
https://docs.google.com/spreadsheets/d/10GhYr_91CANCNKOM_gPy7Sk9hQkTyQGNgNwNgfHeUtI/edit
Generated sheet for pkg/util/...
https://docs.google.com/spreadsheets/d/1yhzKN-A5iwllJqG_Z9VnD7r1gzvwfh_0J8cklginBFE/edit
Generated sheet for pkg/rpc/...
https://docs.google.com/spreadsheets/d/10hoxvxcdiN79fKDfWj6uLO-iemETDbLxCUI0V8xFWuE/edit

Jira issue: CRDB-24434