Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

batches: implement export changeset bulk operation#56721

Merged
BolajiOlajide merged 38 commits into
mainfrom
bo/export-changeset-feature
Oct 31, 2023
Merged

batches: implement export changeset bulk operation#56721
BolajiOlajide merged 38 commits into
mainfrom
bo/export-changeset-feature

Conversation

@BolajiOlajide

@BolajiOlajide BolajiOlajide commented Sep 18, 2023

Copy link
Copy Markdown
Contributor

This PR builds on the existing work done by @topebali in #55850. I made this a separate PR because I wanted to keep #55850 for reference since this PR introduces a new approach for creating data for download via the graphQL API.

We use Go's CSV module to generate and create a CSV based on the changeset IDs passed; this CSV is then returned to the UI as a string. We then take this string and create URL object to make it available for download.

output.mp4

Test plan

  • Added unit tests for the new export changeset functionality.
  • Extensively test with different code hosts and states of changesets.

@cla-bot cla-bot Bot added the cla-signed label Sep 18, 2023
@BolajiOlajide BolajiOlajide changed the title internal/batches: Add export to type and service batches: implement export changeset bulk operation Sep 18, 2023
@BolajiOlajide BolajiOlajide force-pushed the bo/export-changeset-feature branch from 75283a1 to e9b815b Compare September 18, 2023 12:38
@BolajiOlajide BolajiOlajide self-assigned this Sep 21, 2023
@BolajiOlajide BolajiOlajide force-pushed the bo/export-changeset-feature branch 3 times, most recently from bae566e to 00b6fa7 Compare September 22, 2023 14:02
@BolajiOlajide BolajiOlajide marked this pull request as ready for review September 22, 2023 14:16
@BolajiOlajide BolajiOlajide requested review from a team, eseliger and topebali September 22, 2023 14:16
@sourcegraph-bot

sourcegraph-bot commented Sep 22, 2023

Copy link
Copy Markdown
Contributor

Codenotify: Notifying subscribers in CODENOTIFY files for diff 69bb847...5a0bf02.

Notify File(s)
@eseliger client/web/src/enterprise/batches/detail/backend.ts
client/web/src/enterprise/batches/detail/bulk-operations/BulkOperationNode.tsx
client/web/src/enterprise/batches/detail/changesets/ChangesetSelectRow.tsx
client/web/src/enterprise/batches/detail/changesets/ExportChangesetsModal.story.tsx
client/web/src/enterprise/batches/detail/changesets/ExportChangesetsModal.tsx
cmd/frontend/graphqlbackend/batches.go
internal/batches/service/service.go
internal/batches/service/service_test.go
internal/batches/types/changeset_job.go

@sourcegraph-bot

sourcegraph-bot commented Sep 22, 2023

Copy link
Copy Markdown
Contributor

📖 Storybook live preview

@camdencheek camdencheek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments inline, but first a general question: why is the backend side of this needed? It looks like this is just exporting information about all the changesets associated with a batch change, which can already be done with the batchChange { changesets { ... } } GraphQL query. A client can use that GraphQL query to export as whatever format they want, including CSV.

As a general rule, I think it's preferable to have the client format the data differently if that's how they want it. I don't think we want to cement the format of this API to CSV with a fixed set of columns. Instead, if there is information that is not available on the ChangeSet resolver, we should make those fields accessible via GraphQL. That way, the client can choose exactly which fields they care about, can format it however they want for export, and doesn't require downloading a potentially gigantic CSV string stuffed into a JSON reponse (and can instead be streamed with pagination).

We solve this differently for exporting search results, where we construct the CSV in the client. I'd recommend we look at that approach and consider doing something similar.

Comment thread internal/batches/service/service.go Outdated
Comment thread cmd/frontend/graphqlbackend/batches.graphql Outdated
Comment thread internal/batches/service/service.go Outdated
Comment thread internal/batches/service/service.go Outdated
@BolajiOlajide

Copy link
Copy Markdown
Contributor Author

Some comments inline, but first a general question: why is the backend side of this needed? It looks like this is just exporting information about all the changesets associated with a batch change, which can already be done with the batchChange { changesets { ... } } GraphQL query. A client can use that GraphQL query to export as whatever format they want, including CSV.

As a general rule, I think it's preferable to have the client format the data differently if that's how they want it. I don't think we want to cement the format of this API to CSV with a fixed set of columns. Instead, if there is information that is not available on the ChangeSet resolver, we should make those fields accessible via GraphQL. That way, the client can choose exactly which fields they care about, can format it however they want for export, and doesn't require downloading a potentially gigantic CSV string stuffed into a JSON reponse (and can instead be streamed with pagination).

We solve this differently for exporting search results, where we construct the CSV in the client. I'd recommend we look at that approach and consider doing something similar.

This slipped through the cracks of the walls of my notification system.
Good question. The reason I made this happen on the backend is because it follows the signature of the other changeset jobs. But you have a good point on allowing the client flexibility on querying what they want/need.

I'll take a look at the approach for exporting search results. Thanks

@BolajiOlajide BolajiOlajide force-pushed the bo/export-changeset-feature branch from 00b6fa7 to e478a39 Compare October 11, 2023 20:10
@BolajiOlajide BolajiOlajide marked this pull request as draft October 13, 2023 15:50
Comment thread cmd/frontend/internal/batches/resolvers/resolver.go
@BolajiOlajide BolajiOlajide marked this pull request as ready for review October 13, 2023 21:29
@BolajiOlajide

Copy link
Copy Markdown
Contributor Author

@camdencheek This is ready for review once more.

I've moved the data generation to the client side and now support JSON and CSV. Here's a quick walkthrough of what it looks like.

CleanShot.2023-10-13.at.17.30.41.mp4

@BolajiOlajide BolajiOlajide requested a review from a team October 13, 2023 21:38

@camdencheek camdencheek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the slow review! I forgot this one in "pending" 🤦

Just a few non-critical comments inline

Comment on lines +99 to +103
const element = document.createElement('a')
element.download = `batch_change_export.${selectedDataExportType}`
element.href = url
document.body.append(element)
element.click()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove the element after we're done with it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nice catch. I'll remove that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super minor, I think this logic already exists in the DownloadFileButton component in the wildcard, I'm not sure about the context here and can we use it here instead of manual blob creation logic, but just FYI for further cases like this. https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/client/web/src/components/DownloadFileButton.tsx?L15:14

Comment thread cmd/frontend/graphqlbackend/batches.graphql
Comment thread cmd/frontend/internal/batches/resolvers/resolver.go
Comment thread cmd/frontend/internal/batches/resolvers/resolver_test.go
Comment thread internal/batches/service/service.go
@BolajiOlajide BolajiOlajide force-pushed the bo/export-changeset-feature branch from 4e32f5d to 5a0bf02 Compare October 20, 2023 09:46
@BolajiOlajide BolajiOlajide merged commit 06560e3 into main Oct 31, 2023
@BolajiOlajide BolajiOlajide deleted the bo/export-changeset-feature branch October 31, 2023 07:34
vovakulikov pushed a commit that referenced this pull request Dec 12, 2023
Co-authored-by: topebali <topebali97@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants