Skip to content

feat(gcp): Remove codegen#6384

Merged
kodiakhq[bot] merged 3 commits intocloudquery:mainfrom
erezrokah:chore/gcp_no_codegen
Jan 9, 2023
Merged

feat(gcp): Remove codegen#6384
kodiakhq[bot] merged 3 commits intocloudquery:mainfrom
erezrokah:chore/gcp_no_codegen

Conversation

@erezrokah
Copy link
Copy Markdown
Member

@erezrokah erezrokah commented Jan 5, 2023

Summary

This PR removes code gen from GCP (it keeps only service discovery).
Blocked by cloudquery/plugin-sdk#578 and cloudquery/plugin-sdk#579.

While working on this PR I noticed a few cases where we configured the wrong PKs on tables. I'll fix those separately

@cq-bot cq-bot added the gcp label Jan 5, 2023
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 5, 2023 •
edited
Loading

This PR has the following changes to source plugin(s) tables:

  • Table gcp_apikeys_keys: column order changed for name
  • Table gcp_artifactregistry_locations: column order changed for metadata
  • Table gcp_artifactregistry_locations: column order changed for name
  • Table gcp_cloudiot_device_registries: column order changed for id
  • Table gcp_cloudiot_devices: column order changed for id
  • Table gcp_compute_addresses: column order changed for self_link
  • Table gcp_compute_autoscalers: column order changed for self_link
  • Table gcp_compute_backend_services: column order changed for self_link
  • Table gcp_compute_disk_types: column order changed for self_link
  • Table gcp_compute_disks: column order changed for self_link
  • Table gcp_compute_firewalls: column order changed for self_link
  • Table gcp_compute_forwarding_rules: column order changed for self_link
  • Table gcp_compute_images: column order changed for self_link
  • Table gcp_compute_instance_groups: column order changed for self_link
  • Table gcp_compute_instances: column order changed for self_link
  • Table gcp_compute_interconnects: column order changed for self_link
  • Table gcp_compute_networks: column order changed for self_link
  • Table gcp_compute_projects: column order changed for self_link
  • Table gcp_compute_ssl_certificates: column order changed for self_link
  • Table gcp_compute_ssl_policies: column order changed for self_link
  • Table gcp_compute_subnetworks: column order changed for self_link
  • Table gcp_compute_target_http_proxies: column order changed for self_link
  • Table gcp_compute_target_ssl_proxies: column order changed for self_link
  • Table gcp_compute_url_maps: column order changed for self_link
  • Table gcp_compute_vpn_gateways: column order changed for self_link
  • Table gcp_container_clusters: column order changed for self_link
  • Table gcp_dns_managed_zones: column order changed for id
  • Table gcp_dns_policies: column order changed for id
  • Table gcp_iam_roles: column order changed for name
  • Table gcp_iam_service_accounts: column order changed for oauth2_client_id
  • Table gcp_iam_service_accounts: column order changed for unique_id
  • Table gcp_sql_instances: column order changed for self_link
  • Table gcp_sql_users: column order changed for instance
  • Table gcp_sql_users: column order changed for name

@erezrokah erezrokah force-pushed the chore/gcp_no_codegen branch from 7a8b7aa to fe7041d Compare January 5, 2023 16:49
@vercel
Copy link
Copy Markdown

vercel bot commented Jan 5, 2023 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
cloudquery-web 🔄 Building (Inspect) Jan 9, 2023 at 10:05AM (UTC)

@erezrokah erezrokah force-pushed the chore/gcp_no_codegen branch from fe7041d to 3f3ea70 Compare January 5, 2023 17:02
yevgenypats
yevgenypats approved these changes Jan 5, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@yevgenypats yevgenypats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💨

@erezrokah erezrokah force-pushed the chore/gcp_no_codegen branch from 3f3ea70 to 200be71 Compare January 9, 2023 10:05
@erezrokah erezrokah added priority merge automerge Automatically merge once required checks pass labels Jan 9, 2023
@kodiakhq kodiakhq bot merged commit b55b535 into cloudquery:main Jan 9, 2023
This was referenced Jan 9, 2023
Closed
Merged
kodiakhq bot pushed a commit that referenced this pull request Jan 9, 2023
@cq-bot
chore(main): Release plugins-source-gcp v6.0.0 (#6526)
c43dad9 
🤖 I have created a release *beep* *boop*
---


## [6.0.0](plugins-source-gcp-v5.5.0...plugins-source-gcp-v6.0.0) (2023-01-09)


### ⚠ BREAKING CHANGES

* **gcp-resources:** `gcp_iam_roles` column `etag` type changed from `String` to `IntArray`
* **gcp-resources:** `gcp_iam_service_account_keys` column `public_key_data` type changed from `String` to `IntArray` and columns `valid_after_time, valid_before_time` type changed from `String` to `Timestamp`
* **gcp-resources:** `gcp_iam_service_accounts` column `etag` type changed from `String` to `IntArray`

### Features

* **gcp-resources:** Move `gcp_iam_deny_policies` to use `googleapis/google-cloud-go` ([58baafc](58baafc))
* **gcp-resources:** Move `gcp_iam_roles` to use `googleapis/google-cloud-go` ([58baafc](58baafc))
* **gcp-resources:** Move `gcp_iam_service_account_keys` to use `googleapis/google-cloud-go` ([58baafc](58baafc))
* **gcp-resources:** Move `gcp_iam_service_accounts` to use `googleapis/google-cloud-go` ([58baafc](58baafc))
* **gcp:** Remove codegen ([#6384](#6384)) ([b55b535](b55b535))


### Bug Fixes

* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.21.0 ([#6382](#6382)) ([5baea40](5baea40))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.22.0 ([#6516](#6516)) ([b7e4e73](b7e4e73))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.23.0 ([#6522](#6522)) ([ce24f1d](ce24f1d))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
@erezrokah erezrokah mentioned this pull request Jan 9, 2023
Merged
erezrokah added a commit that referenced this pull request Jan 10, 2023
@erezrokah @kodiakhq
fix(gcp-resources)!: Add project_id, instance_name, name as PKs for…
75cf1c5 
… `gcp_bigtableadmin_tables` (#6536)

<!-- 🎉 Thank you for making CloudQuery awesome by submitting a PR 🎉 -->

#### Summary

`gcp_bigtableadmin_tables` had only `project_id` as PK which means we
can only get 1 per project in `overwrite*` modes.
Since the API doesn't provide an Id we need to use the parent
`instance_name` and table name.
The code to get the table name already exists:

https://github.com/cloudquery/cloudquery/blob/60c0f145577881c4850eb82f546da47a1d57ed5c/plugins/source/gcp/resources/services/bigtableadmin/tables_fetch.go#L28
I added it accidentally in
#6384.

This is a breaking change as it relies on newly added columns for PKs

<!--
Use the following steps to ensure your PR is ready to be reviewed

- [ ] Read the [contribution guidelines](../blob/main/CONTRIBUTING.md)
🧑‍🎓
- [ ] Test locally on your own infrastructure
- [ ] Run `go fmt` to format your code 🖊
- [ ] Lint your changes via `golangci-lint run` 🚨 (install golangci-lint
[here](https://golangci-lint.run/usage/install/#local-installation))
- [ ] Update or add tests 🧪
- [ ] Ensure the status checks below are successful ✅
--->

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@disq disq disq approved these changes

+1 more reviewer

@yevgenypats yevgenypats yevgenypats approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@erezrokah @disq @yevgenypats @cq-bot