Skip to content

cli: add support for cockroach dump to operate on UDTs#49808

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rohany:udt-dump
Jun 22, 2020
Merged

cli: add support for cockroach dump to operate on UDTs#49808
craig[bot] merged 1 commit intocockroachdb:masterfrom
rohany:udt-dump

Conversation

@rohany
Copy link
Copy Markdown
Contributor

@rohany rohany commented Jun 2, 2020

Fixes #47765.

This PR enables cockroach dump to dump databases and tables that
contain user defined types.

Release note (cli change): Allow for cockroach dump to dump tables and
databases that contain user defined types.

@rohany rohany requested review from dt and knz June 2, 2020 18:20
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Jun 2, 2020

Wanted to get a preliminary look at this PR while some open questions around cross database type references are resolved.

The work done here is maybe another reason why it would be easier to do more of dump on the server side.

pkg/cli/dump.go Outdated
}

if shouldDumpTypes && dumpCtx.dumpMode != dumpDataOnly {
if _, err := fmt.Fprintf(w, "SET experimental_enable_enums = true;\n"); err != nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For cross-version compatibility, be careful to tolerate dump talking to a server prior to your changes being available.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I have commented elsewhere, including on #28948, that the current architecture is flawed - all this logic should be done server-side, in which case the cross-version compat story is much easier.)

@rohany rohany force-pushed the udt-dump branch 4 times, most recently from a8336f8 to 5912c14 Compare June 8, 2020 18:31
@rohany rohany marked this pull request as ready for review June 8, 2020 18:32
@rohany rohany requested a review from a team June 8, 2020 18:32
@rohany rohany requested a review from a team as a code owner June 8, 2020 18:32
@rohany rohany requested a review from knz June 9, 2020 17:30
Copy link
Copy Markdown
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the PR but this is a bit out of my depth for a quality review right now.
As you hand the review over to another engineer, be mindful to check:

  1. that this code works fine when ran with a server running at a lower version
  2. that the tests exercise the various error cases

Reviewed 1 of 6 files at r2.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @dt, @knz, and @rohany)

@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Jun 15, 2020

@dt can you take a look at this PR. Maybe @jordanlewis could take a look at the enum specific things.

@rohany rohany force-pushed the udt-dump branch 2 times, most recently from 342f299 to 5dd1cb5 Compare June 15, 2020 17:18
@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Jun 17, 2020

Pinging for reviews!

@rohany rohany force-pushed the udt-dump branch 2 times, most recently from 5c7bdb5 to cb15bb1 Compare June 19, 2020 18:39
@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Jun 19, 2020

cc @mjibson for some small sqlsmith changes

@madelynnblue
Copy link
Copy Markdown
Contributor

sqlsmith stuff LGTM

Copy link
Copy Markdown
Contributor

@dt dt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall LGTM though like @knz I'd prefer to move some of the more involved enum rendering logic to crdb_internal.create_statements

Reviewed 2 of 6 files at r2, 8 of 8 files at r3.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @knz and @rohany)


pkg/cli/dump.go, line 187 at r1 (raw file):

Previously, knz (kena) wrote…

(I have commented elsewhere, including on #28948, that the current architecture is flawed - all this logic should be done server-side, in which case the cross-version compat story is much easier.)

this should be fine -- it is emitting a setting that is only known to 20.2+ it isn't sending it to the server. the types are only understood by a new server so it is fine if the setting is too.


pkg/cli/dump.go, line 362 at r1 (raw file):

Previously, rohany (Rohan Yadav) wrote…

heh, yeah I felt that I was taking it a step too far here. I'll extend the internal table I'm referencing to return the necessary information.

yeah, i'd also have preference for the crdb_internal.create_statements table doing more of this for us than baking a lot of logic into the client.


pkg/cmd/roachtest/dump.go, line 1 at r3 (raw file):

// Copyright 2019 The Cockroach Authors.

nit: 2020

Copy link
Copy Markdown
Contributor Author

@rohany rohany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @knz)


pkg/cli/dump.go, line 362 at r1 (raw file):

Previously, dt (David Taylor) wrote…

yeah, i'd also have preference for the crdb_internal.create_statements table doing more of this for us than baking a lot of logic into the client.

I'm not sure what else I can push to the server. Unfortunately, the driver doesn't return arrays as []string, but rather as []byte that needs to be decoded into an array.

@rohany rohany force-pushed the udt-dump branch 2 times, most recently from e8f8d77 to 1584411 Compare June 22, 2020 17:58
Copy link
Copy Markdown
Contributor

@dt dt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @dt and @knz)


pkg/cli/dump.go, line 362 at r1 (raw file):

Previously, rohany (Rohan Yadav) wrote…

I'm not sure what else I can push to the server. Unfortunately, the driver doesn't return arrays as []string, but rather as []byte that needs to be decoded into an array.

oh, sorry, i was looking at wrong revision in reviewable when i thought that -- this LGTM

Fixes cockroachdb#47765.

This PR enables `cockroach dump` to dump databases and tables that
contain user defined types.

Release note (cli change): Allow for `cockroach dump` to dump tables and
databases that contain user defined types.
@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Jun 22, 2020

TFTR!

bors r=dt

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jun 22, 2020

Build succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: user defined types are unsupported in cockroach dump

5 participants