Skip to content

fix: Migrate only tables specified in config#5424

Merged
kodiakhq[bot] merged 4 commits intomainfrom
migrate-only-selected-tables
Dec 6, 2022
Merged

fix: Migrate only tables specified in config#5424
kodiakhq[bot] merged 4 commits intomainfrom
migrate-only-selected-tables

Conversation

@hermanschaaf
Copy link
Copy Markdown
Contributor

@hermanschaaf hermanschaaf commented Dec 6, 2022

This changes the behavior of migrations so that they only run against the tables selected in the config, without touching any other tables. This also means that only the selected tables will get created on first sync, which is important for destinations like CSV and BigQuery. This should be the final step to bring all aspects (sync, migrate, write) in line and have consistent behavior.

Example runs:

$ ../cloudquery/cli/cli migrate bigquery.yml
Loading spec(s) from bigquery.yml
Source aws will migrate 59 tables.
Starting migration for: aws -> [bigquery]
Migration completed successfully.
$ ../cloudquery/cli/cli sync bigquery.yml
Loading spec(s) from bigquery.yml
Source aws will migrate and sync 59 tables.
Starting migration for: aws -> [bigquery]
Migration completed successfully.
Starting sync for: aws -> [bigquery]
Sync completed successfully. Resources: 225, Errors: 0, Panics: 0, Time: 40s

Closes #4715

@vercel
Copy link
Copy Markdown

vercel bot commented Dec 6, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
cloudquery-web ✅ Ready (Inspect) Visit Preview Dec 6, 2022 at 2:18PM (UTC)

@hermanschaaf hermanschaaf added automerge Automatically merge once required checks pass and removed automerge Automatically merge once required checks pass labels Dec 6, 2022
@hermanschaaf hermanschaaf changed the title fix: Migrate only selected tables fix: Migrate only tables specified in config Dec 6, 2022
@kodiakhq kodiakhq bot merged commit 0ff8663 into main Dec 6, 2022
@kodiakhq kodiakhq bot deleted the migrate-only-selected-tables branch December 6, 2022 14:18
kodiakhq bot pushed a commit that referenced this pull request Dec 7, 2022
🤖 I have created a release *beep* *boop*
---


## [2.0.24](cli-v2.0.23...cli-v2.0.24) (2022-12-07)


### Bug Fixes

* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.11.0 ([#5416](#5416)) ([2e7ca35](2e7ca35))
* Migrate only tables specified in config ([#5424](#5424)) ([0ff8663](0ff8663))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate only the tables specified in config

5 participants