Skip to content

feat(DCP-2201): Add collection export command#377

Merged
keyboSlice merged 7 commits intomainfrom
DCP-2201/feat/collection-export
Mar 26, 2026
Merged

feat(DCP-2201): Add collection export command#377
keyboSlice merged 7 commits intomainfrom
DCP-2201/feat/collection-export

Conversation

@keyboSlice
Copy link
Copy Markdown
Contributor

@keyboSlice keyboSlice commented Mar 26, 2026 •
edited
Loading

feat(DCP-2201): add collection export command

cli-collection-export.mp4

Context

The data-collection-tool backend now exposes an async collection export API (DCP-2199 / DCP-2201). When triggered, it generates a ZIP archive containing all submission responses, collection metadata, a README, and any participant-uploaded files. This PR adds the CLI command to drive that flow end-to-end.

What this does

Adds prolific collection export <collection-id> which:

  1. POSTs to POST /collections/{id}/export to initiate a new export job
  2. If the server already has a valid export cached it returns complete immediately and the file is downloaded without polling
  3. Otherwise polls GET /collections/{id}/export/{exportId} every 3 seconds until the job is complete or failed (10-minute timeout)
  4. Downloads the resulting ZIP from the presigned S3 URL and writes it to disk
$ prolific collection export 5f8e3c2a-1d4b-4e6f-9a7c-2b0d8f3e1c5a
Requesting export for collection 5f8e3c2a-1d4b-4e6f-9a7c-2b0d8f3e1c5a...
...
Export ready. Downloading to 5f8e3c2a-1d4b-4e6f-9a7c-2b0d8f3e1c5a-export.zip...
Export saved to 5f8e3c2a-1d4b-4e6f-9a7c-2b0d8f3e1c5a-export.zip

$ prolific collection export 5f8e3c2a-1d4b-4e6f-9a7c-2b0d8f3e1c5a --output /tmp/my-export.zip

API design

The backend API follows RESTful semantics cleanly:

Method Path Purpose
POST /collections/{id}/export Initiate export; returns generating + export_id (202) or complete + url (200 cached)
GET /collections/{id}/export/{exportId} Poll job status; returns generating, complete, or failed

@keyboSlice keyboSlice requested a review from a team as a code owner March 26, 2026 13:15
Copilot AI review requested due to automatic review settings March 26, 2026 13:15
@prolific-snyk
Copy link
Copy Markdown

prolific-snyk commented Mar 26, 2026 •
edited
Loading

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues
✅ Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@keyboSlice keyboSlice added the dct-ready-for-review Triggers Slack notification when PR is ready for review label Mar 26, 2026
Copilot AI reviewed Mar 26, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new prolific collection export <collection-id> CLI subcommand that drives the async collection export flow exposed by the data-collection-tool backend (initiate export job, poll status, then download the resulting ZIP).

Changes:

  • Introduces collection export Cobra command with polling + ZIP download behavior.
  • Extends the API client interface and implementation with initiate/poll export endpoints and response type.
  • Adds gomock support and new command tests for immediate-complete, polling, and failure paths.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
cmd/collection/export.go New export command implementation (initiate, poll, download).
cmd/collection/export_test.go New unit tests covering cached/immediate completion, polling completion, and failure scenarios.
cmd/collection/collection.go Registers export as a subcommand under collection.
client/client.go Adds API interface methods + client implementations for export initiate/status endpoints.
client/responses.go Adds CollectionExportResponse model for export endpoints.
mock_client/mock_client.go Updates gomock client to support new export methods.

dessskris
dessskris previously approved these changes Mar 26, 2026
View reviewed changes
cmd/collection/export.go Outdated
}

if opts.Output == "" {
opts.Output = fmt.Sprintf("%s-export.zip", opts.Args[0])
Copy link
Copy Markdown
Contributor

@dessskris dessskris Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personal pref territory but what do you think about adding a timestamp to the filename? just thinking for people who download the export multiple times 🤣 they'll end up with %s-export(18).zip after a while

Copy link
Copy Markdown
Contributor Author

@keyboSlice keyboSlice Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great suggestion, ty

dessskris
dessskris previously approved these changes Mar 26, 2026
View reviewed changes
@keyboSlice keyboSlice merged commit cfb7192 into main Mar 26, 2026
5 checks passed
@keyboSlice keyboSlice deleted the DCP-2201/feat/collection-export branch March 26, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@dessskris dessskris dessskris approved these changes

Assignees

No one assigned

Labels

dct-ready-for-review Triggers Slack notification when PR is ready for review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@keyboSlice @prolific-snyk @dessskris