Skip to content

cli, ui: dismiss release notes signup banner per environment variable#56437

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
nkodali:signup-banner-env-var
Nov 20, 2020
Merged

cli, ui: dismiss release notes signup banner per environment variable#56437
craig[bot] merged 1 commit intocockroachdb:masterfrom
nkodali:signup-banner-env-var

Conversation

@nkodali
Copy link
Copy Markdown
Collaborator

@nkodali nkodali commented Nov 9, 2020

Previously, the signup banner could only be dismissed manually.
For internal testing purposes, this banner is unnecessary. This
change provides a way to dismiss the signup banner upon start of
a cluster via the cli by setting the environment variable
COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true.

Resolves #46998

Release note: none

@nkodali nkodali requested a review from dhartunian November 9, 2020 15:34
@nkodali nkodali requested a review from a team as a code owner November 9, 2020 15:34
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Collaborator

@dhartunian dhartunian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@nkodali nkodali force-pushed the signup-banner-env-var branch from 4eb0e7f to 66db602 Compare November 9, 2020 17:24
Copy link
Copy Markdown
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @dhartunian and @nkodali)


pkg/cli/start.go, line 607 at r2 (raw file):

			// Configure UI settings.
			cc, _, finish, err := getClientGRPCConn(ctx, serverCfg)

No that is not a valid approach technically. We do not guarantee the admin server to be available for an internal loopback connection in this way.

getClientGRPCConn is meant for use in a client command running in a different process, possibly over the network. Some server configurations will entirely prevent a client connection from being established from the same host.

Here the proper approach is to take the code in pkg/server/admin.go from the SetUIData() method handler, and move it to a differetn (new) function, then export that function, then use that function directly from here.

@nkodali nkodali force-pushed the signup-banner-env-var branch from 66db602 to f69d6bb Compare November 11, 2020 01:28
Copy link
Copy Markdown
Collaborator Author

@nkodali nkodali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @dhartunian and @knz)


pkg/cli/start.go, line 607 at r2 (raw file):

Previously, knz (kena) wrote…

No that is not a valid approach technically. We do not guarantee the admin server to be available for an internal loopback connection in this way.

getClientGRPCConn is meant for use in a client command running in a different process, possibly over the network. Some server configurations will entirely prevent a client connection from being established from the same host.

Here the proper approach is to take the code in pkg/server/admin.go from the SetUIData() method handler, and move it to a differetn (new) function, then export that function, then use that function directly from here.

Done. I did notice some of the other CLI commands (e.g. init, debug, node, quit) get client connections to issue various requests, sometimes even to admin server. Is there something that makes those situations different than this one? Trying to understand when it is safe to get a client connection vs doing what you suggest here and interfacing directly with the server side code. Thanks for the help!

Copy link
Copy Markdown
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks this looks better.

Reviewed 1 of 3 files at r1, 3 of 3 files at r3.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @nkodali)


pkg/cli/start.go, line 607 at r2 (raw file):

Previously, nkodali wrote…

Done. I did notice some of the other CLI commands (e.g. init, debug, node, quit) get client connections to issue various requests, sometimes even to admin server. Is there something that makes those situations different than this one? Trying to understand when it is safe to get a client connection vs doing what you suggest here and interfacing directly with the server side code. Thanks for the help!

These are client commands that run in a different process, and meant to be run over the network. The code here is part of the server process.


pkg/server/admin.go, line 1201 at r3 (raw file):

// This function writes to system.ui using the internal SQL executor of the provided server.
func SetUIData(
	ctx context.Context, s *Server, req *serverpb.SetUIDataRequest,

It's unclear to me why you pass Server as argument here. Why not make this a method on *Server like the other one? You can give it a different name.


pkg/server/admin.go, line 1203 at r3 (raw file):

	ctx context.Context, s *Server, req *serverpb.SetUIDataRequest,
) (*serverpb.SetUIDataResponse, error) {
	ctx = s.AnnotateCtx(ctx)

please move this ctx assignment to the SetUIData() method above - it belongs to RPC requests.


pkg/server/admin.go, line 1204 at r3 (raw file):

) (*serverpb.SetUIDataResponse, error) {
	ctx = s.AnnotateCtx(ctx)
	userName, err := userFromContext(ctx)

Same, the userName belongs to the RPC server. Pass the username as argument to this function/method.

@nkodali nkodali force-pushed the signup-banner-env-var branch from f69d6bb to 32e1802 Compare November 11, 2020 15:00
Copy link
Copy Markdown
Collaborator Author

@nkodali nkodali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @knz)


pkg/server/admin.go, line 1201 at r3 (raw file):

Previously, knz (kena) wrote…

It's unclear to me why you pass Server as argument here. Why not make this a method on *Server like the other one? You can give it a different name.

Done.


pkg/server/admin.go, line 1203 at r3 (raw file):

Previously, knz (kena) wrote…

please move this ctx assignment to the SetUIData() method above - it belongs to RPC requests.

Done.


pkg/server/admin.go, line 1204 at r3 (raw file):

Previously, knz (kena) wrote…

Same, the userName belongs to the RPC server. Pass the username as argument to this function/method.

Done.

Copy link
Copy Markdown
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: thanks!

Reviewed 2 of 2 files at r4.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @knz)

@nkodali
Copy link
Copy Markdown
Collaborator Author

nkodali commented Nov 12, 2020

I'm having some trouble getting all the roachtests to pass. I ran the roachtests from my machine and everything succeeds. I've re-run on TeamCity several times now. The first few times, different tests failed each time, leading to me believe there is some flakiness perhaps? In later runs, acceptance/bank/cluster-recovery test was failing for a few runs in a row. Any suggestions to debug further?

@knz
Copy link
Copy Markdown
Contributor

knz commented Nov 12, 2020

yes that looks like flakiness that's not yours. can you try rebasing once more maybe?

@nkodali nkodali requested review from a team November 12, 2020 14:41
@nkodali nkodali requested a review from a team as a code owner November 12, 2020 14:41
@nkodali nkodali requested review from dt and removed request for a team November 12, 2020 14:41
@nkodali nkodali marked this pull request as draft November 12, 2020 14:45
@knz
Copy link
Copy Markdown
Contributor

knz commented Nov 12, 2020

that rebase went wrong. there's now too much in the pr

@dhartunian dhartunian removed the request for review from a team November 12, 2020 15:06
@nkodali nkodali force-pushed the signup-banner-env-var branch from 3283eab to 55327e5 Compare November 12, 2020 15:18
@nkodali nkodali marked this pull request as ready for review November 12, 2020 15:19
@nkodali nkodali removed the request for review from dt November 12, 2020 15:25
@nkodali nkodali force-pushed the signup-banner-env-var branch from 55327e5 to 7604831 Compare November 12, 2020 16:48
@nkodali
Copy link
Copy Markdown
Collaborator Author

nkodali commented Nov 12, 2020

bors r=knz,dhartunian

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 12, 2020

Build failed (retrying...):

craig bot pushed a commit that referenced this pull request Nov 12, 2020
55325: ui: Transactions link between Sessions and Statements r=dhartunian a=elkmaster

Resolves: #55131, #56244

Release note (admin ui change): Link to the Transactions page is now shown
between the Sessions and Statements links in the left hand navigation. This more
clearly reflects the hierarchy between the 3 concepts.

56346: testcluster: minor logging improvements r=andreimatei a=andreimatei

Log when TestCluster quiescing starts, and add a node log tag to each
node's quiescing ctx so messages from different nodes can be
disambiguated.

Release note: None

56437: cli, ui: dismiss release notes signup banner per environment variable r=knz,dhartunian a=nkodali

Previously, the signup banner could only be dismissed manually.
For internal testing purposes, this banner is unnecessary. This
change provides a way to dismiss the signup banner upon start of
a cluster via the cli by setting the environment variable
COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true.

Resolves #46998

Release note: none

56533: backupccl: add feature flag support for BACKUP, RESTORE r=otan a=angelapwen

Follow-up to the RFC at #55778. This addresses the SRE use case mentioned in #51643 — instead of moving forward with a global denylist as the RFC indicated, we are prototyping feature flags via cluster settings to turn on/off requested features. The first part of the prototype will be done on `BACKUP` and `RESTORE` commands.

See [this doc](https://docs.google.com/document/d/1nZSdcK7YprL0P4TAuseY-mvlYnd82IaJ_ptAQDoWB6o/edit?) for further details. 

Note that the logic test under `ccl/backupccl/testdata/backup-restore/feature-flags` can be tested with the command `make test PKG=./pkg/ccl/backupccl TESTS='TestBackupRestoreDataDriven'`

— Commit message below — 

Adds a cluster setting to toggle a feature flag for the BACKUP and
RESTORE commands off and on; as well as a broad category for
Bulk IO commands. Currently disabling the cluster setting for Bulk
IO will only disable BACKUP and RESTORE jobs, but other types may
be included in this category in the future..

The feature is being introduced to address a Cockroach Cloud SRE
use case: needing  to disable certain categories of features in
case of cluster failure.

Release note (enterprise change): Adds cluster settings to enable/
disable the BACKUP and RESTORE commands. If a user attempts to use
these features while they are disabled, an error indicating that
the database administrator has disabled the feature is surfaced.

Example usage for the database administrator:
SET CLUSTER SETTING feature.bulkio.backup.enabled = FALSE;
SET CLUSTER SETTING feature.bulkio.backup.enabled = TRUE;
SET CLUSTER SETTING feature.bulkio.restore.enabled = FALSE;
SET CLUSTER SETTING feature.bulkio.restore.enabled = TRUE;
SET CLUSTER SETTING feature.bulkio.enabled = FALSE;
SET CLUSTER SETTING feature.bulkio.enabled = TRUE;

56591: ui: fix Overview screen in OSS builds r=dhartunian a=davepacheco

Previously, when using OSS builds (created with `make buildoss`), when
you loading the DB Console in your browser, you'd get "Page Not Found".
The route for the overview page was missing the leading '/'.  This bug
appears to have been introduced in
722c932.

Release note (admin ui change): This fixes a bug where users of
the OSS builds of CockroachDB would see "Page Not Found" when loading
the Console.

56600: roachpb: remove SetInner in favor of MustSetInner r=nvanbenschoten a=tbg

As of a recent commit, `ErrorDetail.SetInner` became unused, and
we can switch to a `MustSetInner` pattern for `ErrorDetail`. Since
the codegen involved is shared with {Request,Response}Union, those
lose the `SetInner` setter as well; we were always asserting on
the returned bool there anyway so this isn't changing anything.

Release note: None



Co-authored-by: Vlad Los <carrott9@gmail.com>
Co-authored-by: Andrei Matei <andrei@cockroachlabs.com>
Co-authored-by: Namrata Kodali <namrata@cockroachlabs.com>
Co-authored-by: angelapwen <angelaw@cockroachlabs.com>
Co-authored-by: Joshua M. Clulow <josh@sysmgr.org>
Co-authored-by: Tobias Grieger <tobias.b.grieger@gmail.com>
nkodali added a commit to nkodali/cockroach that referenced this pull request Nov 12, 2020
Previously, when starting a node using roachprod, the db console
ui would display a release notes signup banner. This banner is not
useful for users of roachprod. We introduced a way to toggle this
via env var in cockroachdb#56437. This commit sets that
env var, COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true for
`cockroach start*` commands issued from roachprod.

Resolves: cockroachdb#46998
See also: cockroachdb#56437

Release note: none
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 13, 2020

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 13, 2020

Build failed (retrying...):

craig bot pushed a commit that referenced this pull request Nov 13, 2020
56373: hlc: introduce synthetic flag on timestamps r=nvanbenschoten a=nvanbenschoten

Informs #52745.
Informs #36431.

This commit introduces an 8-bit `flags` field on the hlc timestamp struct. The flags are used to provide details about the timestamp and its meaning. They do not affect the sort order of Timestamps.

The commit then introduces the first flag: SYNTHETIC. As discussed in #52745, a synthetic timestamp is defined as a timestamp that makes no claim about the value of clocks in the system. While standard timestamps are pulled from HLC clocks and indicate that some node in the system has a clock with a reading equal to or above its value, a synthetic timestamp makes no such indication. By avoiding a connection to "real time", synthetic timestamps can be used to write values at a future time and to indicate that observed timestamps do not apply to such writes for the purposes of tracking causality between the write and its observers. Observed timestamps will be a critical part of implementing non-blocking transactions (#52745) and fixing the interaction between observed timestamps and transaction refreshing (#36431).

The original plan was to reserve the high-order bit in the logical portion of a timestamp as a "synthetic bit". This is how I began implementing things, but was turned off for a few reasons. First, it was fairly subtle and seemed too easy to get wrong. Using a separate field is more explicit and avoids a class of bugs. Second, I began to have serious concerns about how the synthetic bit would impact timestamp ordering. Every timestamp comparison would need to mask out the bit or risk being incorrect. This was even true of the LSM custom comparator. This seemed difficult to get right and seemed particularly concerning since we're planning on marking only some of a transaction's committed values as synthetic to fix #36431, so if we weren't careful, we could get atomicity violations. There were also minor backwards compatibility concerns.

But a separate field is more expensive in theory, so we need to be careful. However, it turns out that a separate field is mostly free in each case that we care about. In memory, the separate field is effectively free because the Timestamp struct was previously 12 bytes but was always padded out to 16 bytes when included as a field in any other struct. This means that the flags field is replacing existing padding. Over the wire, the field will not be included when zero and will use a varint encoding when not zero, so again, it is mostly free. In the engine key encoding, the field is also not included when zero, and takes up only 1 byte when non-zero, so it is mostly free.

----

First three commits from #56477.

@sumeerbhola I'm hoping you can take a look at the engine-level changes in the `introduce synthetic flag on timestamps` commit (4th commit as of the time of writing). I think the key encoding added here makes sense, but want to make sure you're on board. One possible concern is that we introduce a new 13-byte suffix, which means that combined with a 4-byte sequence number (see #41720 (comment)), we'd collide with the 17 byte `engineKeyVersionLockTableLen`.

@tbg do you mind being the primary reviewer here? I think you know the most about the motivations for this change and will have a good sense of whether this is the best way to introduce additional state on timestamps.

56437: cli, ui: dismiss release notes signup banner per environment variable r=knz,dhartunian a=nkodali

Previously, the signup banner could only be dismissed manually.
For internal testing purposes, this banner is unnecessary. This
change provides a way to dismiss the signup banner upon start of
a cluster via the cli by setting the environment variable
COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true.

Resolves #46998

Release note: none

56627: sql: rework SHOW REGIONS to SHOW REGIONS FROM CLUSTER r=ajstorm a=otan

Resolves #56331 

Release note (sql change): SHOW REGIONS functionality is now deferred to
SHOW REGIONS FROM CLUSTER.

Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: Namrata Kodali <namrata@cockroachlabs.com>
Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 13, 2020

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 13, 2020

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 13, 2020

Build failed:

@knz
Copy link
Copy Markdown
Contributor

knz commented Nov 13, 2020

@nkodali I believe we have identified the source of the test flake so this may not be able to merge until we fix that.

@tbg here's an example PR that's blocked on the node restart. I believe that's what you aim to fix with the error refactor?

nkodali added a commit to nkodali/cockroach that referenced this pull request Nov 13, 2020
Previously, when starting a node using roachprod, the db console
ui would display a release notes signup banner. This banner is not
useful for users of roachprod. We introduced a way to toggle this
via env var in cockroachdb#56437. This commit sets that
env var, COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true for
`cockroach start*` commands issued from roachprod.

Resolves: cockroachdb#46998
See also: cockroachdb#56437

Release note: none
Copy link
Copy Markdown
Collaborator Author

@nkodali nkodali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knz thanks for the heads up

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 2 stale) (waiting on @dhartunian and @knz)

@nkodali nkodali force-pushed the signup-banner-env-var branch from 7604831 to dd55f54 Compare November 17, 2020 18:55
@knz
Copy link
Copy Markdown
Contributor

knz commented Nov 18, 2020

could you rebase this today and see what CI thinks about it now? Thanks

Previously, the signup banner could only be dismissed manually.
For internal testing purposes, this banner is unnecessary. This
change provides a way to dismiss the signup banner upon start of
a cluster via the cli by setting the environment variable
COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true.

Resolves cockroachdb#46998

Release note: none
@nkodali nkodali force-pushed the signup-banner-env-var branch from dd55f54 to dd9613c Compare November 18, 2020 16:44
@nkodali
Copy link
Copy Markdown
Collaborator Author

nkodali commented Nov 18, 2020

bors r=knz,dhartunian

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 18, 2020

Build failed:

@knz
Copy link
Copy Markdown
Contributor

knz commented Nov 18, 2020

ok either you are being unlucky, or there is a real bug here underneath.

@bdarnell can we solicit your additional review here? Is there something in this PR that could cause a lot of our acceptance tests to fail with RPC or HTTP errorS?

@nkodali
Copy link
Copy Markdown
Collaborator Author

nkodali commented Nov 20, 2020

bors r=knz,dhartunian

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 20, 2020

Build succeeded:

@craig craig bot merged commit 2d88439 into cockroachdb:master Nov 20, 2020
craig bot pushed a commit that referenced this pull request Nov 20, 2020
56632: roachprod: dismiss signup banner for nodes started by roachprod r=dhartunian,irfansharif a=nkodali

Previously, when starting a node using roachprod, the db console
ui would display a release notes signup banner. This banner is not
useful for users of roachprod. We introduced a way to toggle this
via env var in #56437. This commit sets that
env var, COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true for
`cockroach start*` commands issued from roachprod.

Resolves: #46998
See also: #56437

Release note: none

56913: sqlproxyccl: count successful connection attempts r=spaskob a=spaskob

This counter will be useful to understand how many successful
connections have been cached so far unless we have gone over
the cache size limit of course.

Release note: none.

Co-authored-by: Namrata Kodali <namrata@cockroachlabs.com>
Co-authored-by: Spas Bojanov <spas@cockroachlabs.com>
@nkodali nkodali deleted the signup-banner-env-var branch November 24, 2020 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ui: Allow the release notes signup banner to be disabled by an environment variable

4 participants