Skip to content

roachprod: default --insecure to true#147737

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
srosenberg:sr/roachprod_insecure_default
Jun 27, 2025
Merged

roachprod: default --insecure to true#147737
craig[bot] merged 1 commit intocockroachdb:masterfrom
srosenberg:sr/roachprod_insecure_default

Conversation

@srosenberg
Copy link
Copy Markdown
Member

@srosenberg srosenberg commented Jun 4, 2025

As of [1], [2], roachprod defaulted to
"secure" mode to keep it in one-to-one
correspondence with roachtest. However,
upon further reflection (see an internal thread [3]),
"secure mode" adds an unnecessary friction.

As of this change, roachprod now defaults to
"insecure" mode while roachtest continues to
default to "secure". Further, drtprod also
continues to default to "secure" mode.

[1] #123593
[2] #123826
[3] https://cockroachlabs.slack.com/archives/C023S0V4YEB/p1748888536180379?thread_ts=1724452649.312729&cid=C023S0V4YEB

Epic: none

Release note: None

@srosenberg srosenberg requested a review from a team as a code owner June 4, 2025 03:03
@srosenberg srosenberg requested review from a team, DarrylWong, herkolategan, nameisbhaskar and shailendra-patel and removed request for a team June 4, 2025 03:03
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@srosenberg srosenberg force-pushed the sr/roachprod_insecure_default branch from 9066382 to c2de624 Compare June 4, 2025 03:06
Copy link
Copy Markdown
Contributor

@DarrylWong DarrylWong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM but I think we still want DRT clusters to be secure right?

@srosenberg
Copy link
Copy Markdown
Member Author

Change LGTM but I think we still want DRT clusters to be secure right?

They were previously "insecure" because of the missing wrapper, which enables "secure" via a side-effect [1]. @nameisbhaskar fyi

[1]

isSecure, err = isSecureCluster(cmd)

@DarrylWong
Copy link
Copy Markdown
Contributor

My understanding of the drtprod wrapper is that it's mainly used to call the execute yaml command. The yaml execution itself invokes a roachprod binary which does correctly check isSecureCluster.

My comment was more so referring to the fact the DRP folks may want to update their scripts to pass in a secure flag when creating a cluster, e.g.:

diff --git a/pkg/cmd/drtprod/configs/drt_scale_300.yaml b/pkg/cmd/drtprod/configs/drt_scale_300.yaml
index 3f3bc3955d8..260808e6bd6 100644
--- a/pkg/cmd/drtprod/configs/drt_scale_300.yaml
+++ b/pkg/cmd/drtprod/configs/drt_scale_300.yaml
@@ -28,6 +28,7 @@ targets:
         args:
           - $CLUSTER
         flags:
+          secure: true

or better yet have some configurable global default for DRT clusters.

@srosenberg
Copy link
Copy Markdown
Member Author

My understanding of the drtprod wrapper is that it's mainly used to call the execute yaml command. The yaml execution itself invokes a roachprod binary which does correctly check isSecureCluster.

I don't think it does (see GetYamlProcessor). Prior to this PR, if you run roachprod ... you get "secure" mode by default and "insecure" with drtprod.

Copy link
Copy Markdown
Contributor

@nameisbhaskar nameisbhaskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nameisbhaskar
Copy link
Copy Markdown
Contributor

My understanding of the drtprod wrapper is that it's mainly used to call the execute yaml command. The yaml execution itself invokes a roachprod binary which does correctly check isSecureCluster.

I don't think it does (see GetYamlProcessor). Prior to this PR, if you run roachprod ... you get "secure" mode by default and "insecure" with drtprod.

Yes, we will have to change all the YAML configurations to include this secure flag.

@srosenberg srosenberg force-pushed the sr/roachprod_insecure_default branch from c2de624 to 77dbbb4 Compare June 26, 2025 04:43
@srosenberg
Copy link
Copy Markdown
Member Author

I don't think it does (see GetYamlProcessor). Prior to this PR, if you run roachprod ... you get "secure" mode by default and "insecure" with drtprod.

That wasn't quite right. The YamlProcessor defaults to passing secure=true into roachprod.Run, so we don't really need to update any of the existing yaml. In drtprod/cli.Init we now set COCKROACH_ROACHPROD_INSECURE=false to preserve the "secure" mode as default. Thus, both commands and yaml will continue to default to "secure", whereas roachprod commands will default to "insecure". PTAL!

As of [1], [2], `roachprod` defaulted to
"secure" mode to keep it in one-to-one
correspondence with `roachtest`. However,
upon further reflection (see an internal thread [3]),
"secure mode" adds an unnecessary friction.

As of this change, `roachprod` now defaults to
"insecure" mode while `roachtest` _continues_ to
default to "secure". Further, `drtprod` also
continues to default to "secure" mode.

[1] cockroachdb#123593
[2] cockroachdb#123826
[3] https://cockroachlabs.slack.com/archives/C023S0V4YEB/p1748888536180379?thread_ts=1724452649.312729&cid=C023S0V4YEB

Epic: none
Release note: None
@srosenberg srosenberg force-pushed the sr/roachprod_insecure_default branch from 77dbbb4 to 8f5f0e6 Compare June 26, 2025 22:34
@srosenberg
Copy link
Copy Markdown
Member Author

Smoke Test -- SELECT_PROBABILITY=0.4

@srosenberg
Copy link
Copy Markdown
Member Author

TFTR!

bors r=herkolategan,darrylwong,nameisbhaskar

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jun 27, 2025

This PR was included in a batch that successfully built, but then failed to merge into master (it was a non-fast-forward update). It will be automatically retried.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jun 27, 2025

@craig craig bot merged commit ad5692c into cockroachdb:master Jun 27, 2025
31 of 32 checks passed
golgeek added a commit to golgeek/cockroach that referenced this pull request Aug 1, 2025
Since cockroachdb#147737, to ease testing for engineering teams, roachprod defaults
to insecure clusters. This change has led to security impact for other
entities at CRL.

This patch brings smart selection of secure vs. insecure cluster
depending on the Cloud project the cluster is provisionned in. If the
cluster is provisionned in the engineering's ephemeral GCP project, it
will default to insecure, while all other configurations will default to
secure.

In order to achieve that, and because this depends on the cluster's
configuration that is not available at the CLI package level, the CLI
now keeps track of the user's arguments (--secure, --insecure, default)
and passes the information to the roachprod library.

Epic: none
Release note: None
golgeek added a commit to golgeek/cockroach that referenced this pull request Aug 8, 2025
Since cockroachdb#147737, to ease testing for engineering teams, roachprod defaults
to insecure clusters. This change has led to security impact for other
entities at CRL.

This patch brings smart selection of secure vs. insecure cluster
depending on the Cloud project the cluster is provisionned in. If the
cluster is provisionned in the engineering's ephemeral GCP project, it
will default to insecure, while all other configurations will default to
secure.

In order to achieve that, and because this depends on the cluster's
configuration that is not available at the CLI package level, the CLI
now keeps track of the user's arguments (--secure, --insecure, default)
and passes the information to the roachprod library.

Epic: none
Release note: None
golgeek added a commit to golgeek/cockroach that referenced this pull request Aug 13, 2025
Since cockroachdb#147737, to ease testing for engineering teams, roachprod defaults
to insecure clusters. This change has led to security impact for other
entities at CRL.

This patch brings smart selection of secure vs. insecure cluster
depending on the Cloud project the cluster is provisionned in. If the
cluster is provisionned in the engineering's ephemeral GCP project, it
will default to insecure, while all other configurations will default to
secure.

In order to achieve that, and because this depends on the cluster's
configuration that is not available at the CLI package level, the CLI
now keeps track of the user's arguments (--secure, --insecure, default)
and passes the information to the roachprod library.

Epic: none
Release note: None
craig bot pushed a commit that referenced this pull request Aug 14, 2025
151230: roachprod: default to secure clusters unless GCE cockroach-ephemeral r=herkolategan,srosenberg a=golgeek

Since #147737, to ease testing for engineering teams, roachprod defaults to insecure clusters. This change has led to security impact for other entities at CRL.

This patch brings smart selection of secure vs. insecure cluster depending on the Cloud project the cluster is provisionned in. If the cluster is provisionned in the engineering's ephemeral GCP project, it will default to insecure, while all other configurations will default to secure.

In order to achieve that, and because this depends on the cluster's configuration that is not available at the CLI package level, the CLI now keeps track of the user's arguments (--secure, --insecure, default) and passes the information to the roachprod library.

Epic: none
Release note: None

Co-authored-by: Ludovic Leroux <ludo.leroux@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants