Skip to content

roachtest: use crdb's workload for fixtures import tpcc#55050

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/tpccWorkloadFix
Sep 30, 2020
Merged

roachtest: use crdb's workload for fixtures import tpcc#55050
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/tpccWorkloadFix

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Sep 30, 2020

Fixes #55042.
Fixes #55041.
Fixes #55039.
Fixes #55038.
Fixes #55037.
Fixes #55036.
Fixes #55035.
Fixes #55033.
Fixes #55029.
Fixes #55024.
Fixes #55022.
Fixes #55020.
Fixes #55019.
Fixes #55018.
Fixes #55017.
Fixes #55016.
Fixes #55013.
Fixes #55010.
Fixes #55009.
Fixes #55008.
Fixes #55003.
Fixes #55002.
Fixes #54998.
Fixes #54995.
Fixes #54822.
Fixes #52693.
Fixes #54802.

We were already doing this in some places, but needed it in others.

@nvb nvb requested a review from tbg September 30, 2020 13:53
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Sep 30, 2020

I want to double-check that I got this right, so I'm going to spin up a few runs of this before pushing it through.

@nvb nvb force-pushed the nvanbenschoten/tpccWorkloadFix branch from fae1e1d to 0a95a12 Compare September 30, 2020 13:56
Copy link
Copy Markdown
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epic "Closes" list.

@nvb nvb force-pushed the nvanbenschoten/tpccWorkloadFix branch from 0a95a12 to 20fc710 Compare September 30, 2020 14:08
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Sep 30, 2020

Looks good with master and release-19.1 crdb binaries.

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 30, 2020

Build succeeded:

@craig craig bot merged commit b2c6c13 into cockroachdb:master Sep 30, 2020
@nvb nvb deleted the nvanbenschoten/tpccWorkloadFix branch October 1, 2020 04:08
nvb added a commit to nvb/cockroach that referenced this pull request Oct 26, 2020
Fixes cockroachdb#55541.
Fixes cockroachdb#55580.

In cockroachdb#54880 and cockroachdb#55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.
craig bot pushed a commit that referenced this pull request Oct 27, 2020
55877: rowflow: account for more memory usage of the row buffer r=yuzefovich a=yuzefovich

Previously, we accounted for the memory usage of the row buffer from
which the rows are pushed from in the routers only when copying the rows
from the row container, but we also have another in-memory row buffer
that we can move the rows from which was missing the memory accounting.
This is now fixed which also deflakes the router disk spill test.

Fixes: #55848.

Release note: None

55907: sql: remove vectorize=201auto option r=yuzefovich a=yuzefovich

`201auto` option of `vectorize` setting has been removed since it no
longer makes sense to keep (it was introduced as an escape hatch for
20.2 release). Note that we don't need to bump the distsql version
because of this change since it is backwards compatible - if the gateway
is running the old version that has `vectorize=201auto` set, then we
will check whether flows for all nodes don't have non-streaming
operators and possibly use `off` option on the flow setup request, then
the newer version remote node will check the vectorize setting on the
request whether it is not `off` and setup the vectorized flow if it is
not.

Release note (sql change): `201auto` value for `vectorize` session
variable and the corresponding cluster setting has been removed.

55982: roachtest: use IMPORT for all TPC-C tests r=nvanbenschoten a=nvanbenschoten

Fixes #55541.
Fixes #55580.

In #54880 and #55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.

55994: settings: delete StateMachineSetting, introduce VersionSetting r=irfansharif a=irfansharif

We introduced the custom StateMachineSetting type in #17216 to power the
underlying machinery for version upgrades:

  SET CLUSTER SETTING version = <major>-<minor>;

At the time we left it generalizable enough to make room for future
settings with arbitrary internal transitions. For cluster versions this
meant only allowing transitions from one major version to the immediate
next, all the while making sure that each binary in the cluster was able
to activate the targeted version (the proposed version fell within the
allowable range ["binary min supported version", "binary version"]).

In the three years since we haven't added any state-machine validated
settings that fit the bill, and the layers of abstractions to get to
validated cluster version updates are one too many for (my) comfort.
This PR introduces a new VersionSetting type that is more tightly
coupled with the ClusterVersion type itself.

VersionSetting uses language that only appropriate for cluster versions,
hopefully clarifying things as we go. We'll depend on this clarification
in future PRs when we remove the use of gossip in disseminating cluster
version bumps.

Release note (sql/cli change): The underlying type for the version
cluster setting has been changed. Previously it was of an internal type
representing "state machine", but now it's simply "version". This has no
operational implications, but it does reflect differently in a few
spots:
  - The `Type` column in `cockroach gen settings-list` will now show
    "version" instead of "custom validation"
  - The `setting_type` column for `version` in `SHOW CLUSTER SETTINGS`
    will now show a "v" instead of an "m"
  - The `valueType` column for `version` in `system.settings` will now
    show a "v" instead of an "m"


56006: logictest: deflake a test r=yuzefovich a=yuzefovich

We've recently merged a test that in very rare circumstances could
produce a float result that differs from the expected one by 1 in the
15th significant digit (after rounding). I believe that could occur,
e.g. when the 15th and 16th significant digits were `35`, and we matched
the spec of supporting 15 significant digits for floats, yet the
rounding makes us return an unexpected result. This commit rounds to the
precision of 1 digit less which should make the test non-flaky.

Release note: None

Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
nvb added a commit to nvb/cockroach that referenced this pull request Nov 9, 2020
Fixes cockroachdb#55541.
Fixes cockroachdb#55580.

In cockroachdb#54880 and cockroachdb#55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.
nvb added a commit to nvb/cockroach that referenced this pull request Dec 1, 2020
Fixes cockroachdb#55541.
Fixes cockroachdb#55580.

In cockroachdb#54880 and cockroachdb#55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment