feat: Added store fetch summary routine#356
Conversation
roneli
left a comment
There was a problem hiding this comment.
Didn't post this review for some reason, sorry for the delay. Looks like a good start, have some structural changes to purpose + we need to support migrate for core tables that are created, otherwise it will be hard to manage it.
extended fetch summary structure changelog
Co-authored-by: Kemal <223029+disq@users.noreply.github.com>
roneli
left a comment
There was a problem hiding this comment.
Looking good, had a few extra comments
# Conflicts: # pkg/client/client.go
# Conflicts: # pkg/client/migrations/1_v0.17.4.down.sql
yevgenypats
left a comment
There was a problem hiding this comment.
I would run the migration on fetch and not on new Client because otherwise we will have to have a database running even for commands that do not require a database such as cloudquery policy describe and potentially others.
I would add there are some places that client can be initialized without dsn otherwise it tries to connect, I think also in policy describe it tries. So if dsn != "" its safe to run migrations |
roneli
left a comment
There was a problem hiding this comment.
More comments I missed in prior review
Co-authored-by: Ron <38083777+roneli@users.noreply.github.com>
Co-authored-by: Ron <38083777+roneli@users.noreply.github.com>
# Conflicts: # go.mod # go.sum
If we will have some other core tables related to other commands it is better to migrate in the client initialization |
Co-authored-by: Ron <38083777+roneli@users.noreply.github.com>
Co-authored-by: Ron <38083777+roneli@users.noreply.github.com>
* upstream/main: chore: Synced local '.github/workflows/' with remote 'workflows/common' (cloudquery#421) fix: Don't show telemetry notice when it's not enabled (cloudquery#418) feat: Increase ulimit in unix environment (cloudquery#416) feat: Support build-schema for all providers in config (cloudquery#414) fix: Fetch summary fixes (cloudquery#417) feat: Added store fetch summary routine (cloudquery#356) chore: Synced file(s) with cloudquery/.github (cloudquery#415) feat: Expose max_parallel_resource_fetch_limit (cloudquery#413) feat: Adjust log messages when .cq dir is absent (cloudquery#411) fix: Adjust policy describe message (cloudquery#409) fix: Exit with 1 on policy error (cloudquery#412)
🤖 I have created a release *beep* *boop* --- ## [0.11.11](cloudquery/cq-provider-azure@v0.11.10...v0.11.11) (2022-06-22) ### Features * YAML config support ([#353](cloudquery/cq-provider-azure#353)) ([045e92f](cloudquery/cq-provider-azure@045e92f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [0.8.15](cloudquery/cq-provider-gcp@v0.8.14...v0.8.15) (2022-06-22) ### Features * YAML config support ([#352](cloudquery/cq-provider-gcp#352)) ([23ddfb4](cloudquery/cq-provider-gcp@23ddfb4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
closes #284