Skip to content

roachtest: use fixtures import for tpcc, regardless of cloud#54880

Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom
nvb:nvanbenschoten/importTPCC
Sep 29, 2020
Merged

roachtest: use fixtures import for tpcc, regardless of cloud#54880
craig[bot] merged 2 commits intocockroachdb:masterfrom
nvb:nvanbenschoten/importTPCC

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Sep 28, 2020

Up to this point, we had run fixtures load (RESTORE) on gce and
fixtures import (IMPORT) on aws and azure. This was confusing,
inconsistent, and made the process of updating the TPC-C workload more
effort than strictly necessary. Now that IMPORT is stable and just as
fast as, if not faster than, RESTORE, let's use that everywhere.

@nvb nvb requested a review from RaduBerinde September 28, 2020 18:39
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!!

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Sep 28, 2020

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 29, 2020

Merge conflict.

@RaduBerinde
Copy link
Copy Markdown
Member

Ah, sorry for the conflict. I bors-ed it after you but I guess that doesn't matter..

nvb added 2 commits September 28, 2020 21:54
These were pretty outdated and the test was running slow, as a result.

Release justification: testing only
Up to this point, we had run `fixtures load` (RESTORE) on gce and
`fixtures import` (IMPORT) on aws and azure. This was confusing,
inconsistent, and made the process of updating the TPC-C workload more
effort than strictly necessary. Now that IMPORT is stable and just as
fast as, if not faster than, RESTORE, let's use that everywhere.

Release justification: testing only
@nvb nvb force-pushed the nvanbenschoten/importTPCC branch from 8c496b7 to abda03b Compare September 29, 2020 01:56
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Sep 29, 2020

No problem, easy rebase.

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 29, 2020

Build succeeded:

@craig craig bot merged commit 77dd840 into cockroachdb:master Sep 29, 2020
@nvb nvb deleted the nvanbenschoten/importTPCC branch September 30, 2020 13:54
nvb added a commit to nvb/cockroach that referenced this pull request Oct 26, 2020
Fixes cockroachdb#55541.
Fixes cockroachdb#55580.

In cockroachdb#54880 and cockroachdb#55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.
craig bot pushed a commit that referenced this pull request Oct 27, 2020
55877: rowflow: account for more memory usage of the row buffer r=yuzefovich a=yuzefovich

Previously, we accounted for the memory usage of the row buffer from
which the rows are pushed from in the routers only when copying the rows
from the row container, but we also have another in-memory row buffer
that we can move the rows from which was missing the memory accounting.
This is now fixed which also deflakes the router disk spill test.

Fixes: #55848.

Release note: None

55907: sql: remove vectorize=201auto option r=yuzefovich a=yuzefovich

`201auto` option of `vectorize` setting has been removed since it no
longer makes sense to keep (it was introduced as an escape hatch for
20.2 release). Note that we don't need to bump the distsql version
because of this change since it is backwards compatible - if the gateway
is running the old version that has `vectorize=201auto` set, then we
will check whether flows for all nodes don't have non-streaming
operators and possibly use `off` option on the flow setup request, then
the newer version remote node will check the vectorize setting on the
request whether it is not `off` and setup the vectorized flow if it is
not.

Release note (sql change): `201auto` value for `vectorize` session
variable and the corresponding cluster setting has been removed.

55982: roachtest: use IMPORT for all TPC-C tests r=nvanbenschoten a=nvanbenschoten

Fixes #55541.
Fixes #55580.

In #54880 and #55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.

55994: settings: delete StateMachineSetting, introduce VersionSetting r=irfansharif a=irfansharif

We introduced the custom StateMachineSetting type in #17216 to power the
underlying machinery for version upgrades:

  SET CLUSTER SETTING version = <major>-<minor>;

At the time we left it generalizable enough to make room for future
settings with arbitrary internal transitions. For cluster versions this
meant only allowing transitions from one major version to the immediate
next, all the while making sure that each binary in the cluster was able
to activate the targeted version (the proposed version fell within the
allowable range ["binary min supported version", "binary version"]).

In the three years since we haven't added any state-machine validated
settings that fit the bill, and the layers of abstractions to get to
validated cluster version updates are one too many for (my) comfort.
This PR introduces a new VersionSetting type that is more tightly
coupled with the ClusterVersion type itself.

VersionSetting uses language that only appropriate for cluster versions,
hopefully clarifying things as we go. We'll depend on this clarification
in future PRs when we remove the use of gossip in disseminating cluster
version bumps.

Release note (sql/cli change): The underlying type for the version
cluster setting has been changed. Previously it was of an internal type
representing "state machine", but now it's simply "version". This has no
operational implications, but it does reflect differently in a few
spots:
  - The `Type` column in `cockroach gen settings-list` will now show
    "version" instead of "custom validation"
  - The `setting_type` column for `version` in `SHOW CLUSTER SETTINGS`
    will now show a "v" instead of an "m"
  - The `valueType` column for `version` in `system.settings` will now
    show a "v" instead of an "m"


56006: logictest: deflake a test r=yuzefovich a=yuzefovich

We've recently merged a test that in very rare circumstances could
produce a float result that differs from the expected one by 1 in the
15th significant digit (after rounding). I believe that could occur,
e.g. when the 15th and 16th significant digits were `35`, and we matched
the spec of supporting 15 significant digits for floats, yet the
rounding makes us return an unexpected result. This commit rounds to the
precision of 1 digit less which should make the test non-flaky.

Release note: None

Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
nvb added a commit to nvb/cockroach that referenced this pull request Nov 9, 2020
Fixes cockroachdb#55541.
Fixes cockroachdb#55580.

In cockroachdb#54880 and cockroachdb#55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.
nvb added a commit to nvb/cockroach that referenced this pull request Dec 1, 2020
Fixes cockroachdb#55541.
Fixes cockroachdb#55580.

In cockroachdb#54880 and cockroachdb#55050, we switched to using IMPORT for most TPC-C tests,
in favor of BACKUP, which requires fixtures to be regenerated whenever
we changed the workload. In this PR, we switch over the remaining uses
of `fixtures load tpcc` to `fixtures import tpcc` to avoid compatibility
issues on older release branches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants