Skip to content

Commit aefaa9a

Browse files
erezrokaherezrokah
authored
fix(aws-code-gen): Prefix service name when filtering relations from table list (#3546)
<!-- 🎉 Thank you for making CloudQuery awesome by submitting a PR 🎉 --> #### Summary We're missing a bunch of top level tables, as if there's a relation function named "Instances" it will also filter a top level "Instances" service. See tables for v4.0.0: https://github.com/cloudquery/cloudquery/blob/plugins-source-aws-v4.0.0/plugins/source/aws/resources/plugin/tables.go#L145 See tables for v4.2.1: https://github.com/cloudquery/cloudquery/blob/plugins-source-aws-v4.2.1/plugins/source/aws/resources/plugin/tables.go#L146 <!-- Use the following steps to ensure your PR is ready to be reviewed - [ ] Read the [contribution guidelines](../blob/main/CONTRIBUTING.md) 🧑‍🎓 - [ ] Test locally on your own infrastructure - [ ] Run `go fmt` to format your code 🖊 - [ ] Lint your changes via `golangci-lint run` 🚨 (install golangci-lint [here](https://golangci-lint.run/usage/install/#local-installation)) - [ ] Update or add tests 🧪 - [ ] Ensure the status checks below are successful ✅ --->
1 parent a77c1b9 commit aefaa9a

22 files changed

+751
-3
lines changed

‎plugins/source/aws/codegen/tables/generate.go‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,15 @@ import (
44
"bytes"
55
"embed"
66
"fmt"
7-
"github.com/iancoleman/strcase"
87
"go/format"
98
"os"
109
"path"
1110
"runtime"
1211
"strings"
1312
"text/template"
1413

14+
"github.com/iancoleman/strcase"
15+
1516
"github.com/cloudquery/cloudquery/plugins/source/aws/codegen/recipes"
1617
)
1718

@@ -57,11 +58,11 @@ func removeChildResources(resources []*recipes.Resource) []*recipes.Resource {
5758
relations := map[string]bool{}
5859
for _, r := range resources {
5960
for _, rel := range r.Relations {
60-
relations[strings.TrimSuffix(rel, "()")] = true
61+
relations[r.Service+"."+strings.TrimSuffix(rel, "()")] = true
6162
}
6263
}
6364
for _, r := range resources {
64-
funcName := strcase.ToCamel(r.SubService)
65+
funcName := r.Service + "." + strcase.ToCamel(r.SubService)
6566
if relations[funcName] {
6667
continue
6768
}

‎plugins/source/aws/docs/tables/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@
8787
| [aws_docdb_clusters](aws_docdb_clusters.md) |
8888
| ↳ [aws_docdb_cluster_snapshots](aws_docdb_cluster_snapshots.md) |
8989
| ↳ [aws_docdb_instances](aws_docdb_instances.md) |
90+
| [aws_docdb_cluster_parameter_groups](aws_docdb_cluster_parameter_groups.md) |
9091
| [aws_docdb_certificates](aws_docdb_certificates.md) |
9192
| [aws_docdb_engine_versions](aws_docdb_engine_versions.md) |
9293
| [aws_docdb_subnet_groups](aws_docdb_subnet_groups.md) |
@@ -103,6 +104,7 @@
103104
| [aws_ec2_hosts](aws_ec2_hosts.md) |
104105
| [aws_ec2_images](aws_ec2_images.md) |
105106
| [aws_ec2_instance_statuses](aws_ec2_instance_statuses.md) |
107+
| [aws_ec2_instances](aws_ec2_instances.md) |
106108
| [aws_ec2_instance_types](aws_ec2_instance_types.md) |
107109
| [aws_ec2_internet_gateways](aws_ec2_internet_gateways.md) |
108110
| [aws_ec2_key_pairs](aws_ec2_key_pairs.md) |
@@ -148,6 +150,8 @@
148150
| [aws_elasticache_reserved_cache_nodes_offerings](aws_elasticache_reserved_cache_nodes_offerings.md) |
149151
| [aws_elasticache_reserved_cache_nodes](aws_elasticache_reserved_cache_nodes.md) |
150152
| [aws_elasticache_service_updates](aws_elasticache_service_updates.md) |
153+
| [aws_elasticache_snapshots](aws_elasticache_snapshots.md) |
154+
| [aws_elasticache_user_groups](aws_elasticache_user_groups.md) |
151155
| [aws_elasticache_users](aws_elasticache_users.md) |
152156
| [aws_elasticbeanstalk_application_versions](aws_elasticbeanstalk_application_versions.md) |
153157
| [aws_elasticbeanstalk_applications](aws_elasticbeanstalk_applications.md) |
@@ -189,6 +193,7 @@
189193
| [aws_fsx_data_repository_associations](aws_fsx_data_repository_associations.md) |
190194
| [aws_fsx_data_repository_tasks](aws_fsx_data_repository_tasks.md) |
191195
| [aws_fsx_file_systems](aws_fsx_file_systems.md) |
196+
| [aws_fsx_snapshots](aws_fsx_snapshots.md) |
192197
| [aws_fsx_storage_virtual_machines](aws_fsx_storage_virtual_machines.md) |
193198
| [aws_fsx_volumes](aws_fsx_volumes.md) |
194199
| [aws_glacier_vaults](aws_glacier_vaults.md) |
@@ -272,30 +277,40 @@
272277
| ↳ [aws_lightsail_disk_snapshot](aws_lightsail_disk_snapshot.md) |
273278
| [aws_lightsail_distributions](aws_lightsail_distributions.md) |
274279
| [aws_lightsail_instance_snapshots](aws_lightsail_instance_snapshots.md) |
280+
| [aws_lightsail_instances](aws_lightsail_instances.md) |
281+
| ↳ [aws_lightsail_instance_port_states](aws_lightsail_instance_port_states.md) |
275282
| [aws_lightsail_load_balancers](aws_lightsail_load_balancers.md) |
276283
| ↳ [aws_lightsail_load_balancer_tls_certificates](aws_lightsail_load_balancer_tls_certificates.md) |
277284
| [aws_lightsail_static_ips](aws_lightsail_static_ips.md) |
278285
| [aws_mq_brokers](aws_mq_brokers.md) |
279286
| ↳ [aws_mq_broker_configurations](aws_mq_broker_configurations.md) |
280287
| ↳ [aws_mq_broker_users](aws_mq_broker_users.md) |
281288
| [aws_mwaa_environments](aws_mwaa_environments.md) |
289+
| [aws_neptune_cluster_parameter_groups](aws_neptune_cluster_parameter_groups.md) |
290+
| ↳ [aws_neptune_cluster_parameter_group_parameters](aws_neptune_cluster_parameter_group_parameters.md) |
291+
| [aws_neptune_cluster_snapshots](aws_neptune_cluster_snapshots.md) |
282292
| [aws_neptune_clusters](aws_neptune_clusters.md) |
283293
| [aws_neptune_db_parameter_groups](aws_neptune_db_parameter_groups.md) |
284294
| ↳ [aws_neptune_db_parameter_group_db_parameters](aws_neptune_db_parameter_group_db_parameters.md) |
285295
| [aws_neptune_global_clusters](aws_neptune_global_clusters.md) |
286296
| [aws_neptune_event_subscriptions](aws_neptune_event_subscriptions.md) |
297+
| [aws_neptune_instances](aws_neptune_instances.md) |
287298
| [aws_neptune_subnet_groups](aws_neptune_subnet_groups.md) |
288299
| [aws_organizations_accounts](aws_organizations_accounts.md) |
289300
| [aws_qldb_ledgers](aws_qldb_ledgers.md) |
290301
| ↳ [aws_qldb_ledger_journal_kinesis_streams](aws_qldb_ledger_journal_kinesis_streams.md) |
291302
| ↳ [aws_qldb_ledger_journal_s3_exports](aws_qldb_ledger_journal_s3_exports.md) |
292303
| [aws_rds_certificates](aws_rds_certificates.md) |
304+
| [aws_rds_cluster_parameter_groups](aws_rds_cluster_parameter_groups.md) |
305+
| ↳ [aws_rds_cluster_parameter_group_parameters](aws_rds_cluster_parameter_group_parameters.md) |
306+
| [aws_rds_cluster_snapshots](aws_rds_cluster_snapshots.md) |
293307
| [aws_rds_clusters](aws_rds_clusters.md) |
294308
| [aws_rds_db_parameter_groups](aws_rds_db_parameter_groups.md) |
295309
| ↳ [aws_rds_db_parameter_group_db_parameters](aws_rds_db_parameter_group_db_parameters.md) |
296310
| [aws_rds_db_security_groups](aws_rds_db_security_groups.md) |
297311
| [aws_rds_db_snapshots](aws_rds_db_snapshots.md) |
298312
| [aws_rds_event_subscriptions](aws_rds_event_subscriptions.md) |
313+
| [aws_rds_instances](aws_rds_instances.md) |
299314
| [aws_rds_subnet_groups](aws_rds_subnet_groups.md) |
300315
| [aws_redshift_clusters](aws_redshift_clusters.md) |
301316
| ↳ [aws_redshift_snapshots](aws_redshift_snapshots.md) |
@@ -341,12 +356,16 @@
341356
| [aws_sns_topics](aws_sns_topics.md) |
342357
| [aws_sqs_queues](aws_sqs_queues.md) |
343358
| [aws_ssm_documents](aws_ssm_documents.md) |
359+
| [aws_ssm_instances](aws_ssm_instances.md) |
360+
| ↳ [aws_ssm_instance_compliance_items](aws_ssm_instance_compliance_items.md) |
344361
| [aws_ssm_parameters](aws_ssm_parameters.md) |
345362
| [aws_transfer_servers](aws_transfer_servers.md) |
346363
| [aws_wafregional_rate_based_rules](aws_wafregional_rate_based_rules.md) |
347364
| [aws_wafregional_rule_groups](aws_wafregional_rule_groups.md) |
365+
| [aws_wafregional_rules](aws_wafregional_rules.md) |
348366
| [aws_wafregional_web_acls](aws_wafregional_web_acls.md) |
349367
| [aws_waf_rule_groups](aws_waf_rule_groups.md) |
368+
| [aws_waf_rules](aws_waf_rules.md) |
350369
| [aws_waf_subscribed_rule_groups](aws_waf_subscribed_rule_groups.md) |
351370
| [aws_waf_web_acls](aws_waf_web_acls.md) |
352371
| [aws_wafv2_ipsets](aws_wafv2_ipsets.md) |

‎plugins/source/aws/docs/tables/aws_docdb_cluster_parameter_groups.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Table: aws_docdb_cluster_parameter_groups
2+
3+
https://docs.aws.amazon.com/documentdb/latest/developerguide/API_DBClusterParameterGroup.html
4+
5+
The primary key for this table is **arn**.
6+
7+
8+
## Columns
9+
| Name | Type |
10+
| ------------- | ------------- |
11+
|_cq_source_name|String|
12+
|_cq_sync_time|Timestamp|
13+
|_cq_id|UUID|
14+
|_cq_parent_id|UUID|
15+
|account_id|String|
16+
|region|String|
17+
|tags|JSON|
18+
|arn (PK)|String|
19+
|parameters|JSON|
20+
|db_cluster_parameter_group_name|String|
21+
|db_parameter_group_family|String|
22+
|description|String|

‎plugins/source/aws/docs/tables/aws_ec2_instances.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Table: aws_ec2_instances
2+
3+
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_Instance.html
4+
5+
The primary key for this table is **arn**.
6+
7+
8+
## Columns
9+
| Name | Type |
10+
| ------------- | ------------- |
11+
|_cq_source_name|String|
12+
|_cq_sync_time|Timestamp|
13+
|_cq_id|UUID|
14+
|_cq_parent_id|UUID|
15+
|account_id|String|
16+
|region|String|
17+
|arn (PK)|String|
18+
|state_transition_reason_time|Timestamp|
19+
|ami_launch_index|Int|
20+
|architecture|String|
21+
|block_device_mappings|JSON|
22+
|boot_mode|String|
23+
|capacity_reservation_id|String|
24+
|capacity_reservation_specification|JSON|
25+
|client_token|String|
26+
|cpu_options|JSON|
27+
|ebs_optimized|Bool|
28+
|elastic_gpu_associations|JSON|
29+
|elastic_inference_accelerator_associations|JSON|
30+
|ena_support|Bool|
31+
|enclave_options|JSON|
32+
|hibernation_options|JSON|
33+
|hypervisor|String|
34+
|iam_instance_profile|JSON|
35+
|image_id|String|
36+
|instance_id|String|
37+
|instance_lifecycle|String|
38+
|instance_type|String|
39+
|ipv6_address|String|
40+
|kernel_id|String|
41+
|key_name|String|
42+
|launch_time|Timestamp|
43+
|licenses|JSON|
44+
|maintenance_options|JSON|
45+
|metadata_options|JSON|
46+
|monitoring|JSON|
47+
|network_interfaces|JSON|
48+
|outpost_arn|String|
49+
|placement|JSON|
50+
|platform|String|
51+
|platform_details|String|
52+
|private_dns_name|String|
53+
|private_dns_name_options|JSON|
54+
|private_ip_address|String|
55+
|product_codes|JSON|
56+
|public_dns_name|String|
57+
|public_ip_address|String|
58+
|ramdisk_id|String|
59+
|root_device_name|String|
60+
|root_device_type|String|
61+
|security_groups|JSON|
62+
|source_dest_check|Bool|
63+
|spot_instance_request_id|String|
64+
|sriov_net_support|String|
65+
|state|JSON|
66+
|state_reason|JSON|
67+
|state_transition_reason|String|
68+
|subnet_id|String|
69+
|tags|JSON|
70+
|tpm_support|String|
71+
|usage_operation|String|
72+
|usage_operation_update_time|Timestamp|
73+
|virtualization_type|String|
74+
|vpc_id|String|

‎plugins/source/aws/docs/tables/aws_elasticache_snapshots.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Table: aws_elasticache_snapshots
2+
3+
https://docs.aws.amazon.com/AmazonElastiCache/latest/APIReference/API_Snapshot.html
4+
5+
The primary key for this table is **arn**.
6+
7+
8+
## Columns
9+
| Name | Type |
10+
| ------------- | ------------- |
11+
|_cq_source_name|String|
12+
|_cq_sync_time|Timestamp|
13+
|_cq_id|UUID|
14+
|_cq_parent_id|UUID|
15+
|account_id|String|
16+
|region|String|
17+
|arn (PK)|String|
18+
|auto_minor_version_upgrade|Bool|
19+
|automatic_failover|String|
20+
|cache_cluster_create_time|Timestamp|
21+
|cache_cluster_id|String|
22+
|cache_node_type|String|
23+
|cache_parameter_group_name|String|
24+
|cache_subnet_group_name|String|
25+
|data_tiering|String|
26+
|engine|String|
27+
|engine_version|String|
28+
|kms_key_id|String|
29+
|node_snapshots|JSON|
30+
|num_cache_nodes|Int|
31+
|num_node_groups|Int|
32+
|port|Int|
33+
|preferred_availability_zone|String|
34+
|preferred_maintenance_window|String|
35+
|preferred_outpost_arn|String|
36+
|replication_group_description|String|
37+
|replication_group_id|String|
38+
|snapshot_name|String|
39+
|snapshot_retention_limit|Int|
40+
|snapshot_source|String|
41+
|snapshot_status|String|
42+
|snapshot_window|String|
43+
|topic_arn|String|
44+
|vpc_id|String|

‎plugins/source/aws/docs/tables/aws_elasticache_user_groups.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Table: aws_elasticache_user_groups
2+
3+
https://docs.aws.amazon.com/AmazonElastiCache/latest/APIReference/API_UserGroup.html
4+
5+
The primary key for this table is **arn**.
6+
7+
8+
## Columns
9+
| Name | Type |
10+
| ------------- | ------------- |
11+
|_cq_source_name|String|
12+
|_cq_sync_time|Timestamp|
13+
|_cq_id|UUID|
14+
|_cq_parent_id|UUID|
15+
|account_id|String|
16+
|region|String|
17+
|arn (PK)|String|
18+
|engine|String|
19+
|minimum_engine_version|String|
20+
|pending_changes|JSON|
21+
|replication_groups|StringArray|
22+
|status|String|
23+
|user_group_id|String|
24+
|user_ids|StringArray|

‎plugins/source/aws/docs/tables/aws_fsx_snapshots.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Table: aws_fsx_snapshots
2+
3+
https://docs.aws.amazon.com/fsx/latest/APIReference/API_Snapshot.html
4+
5+
The primary key for this table is **arn**.
6+
7+
8+
## Columns
9+
| Name | Type |
10+
| ------------- | ------------- |
11+
|_cq_source_name|String|
12+
|_cq_sync_time|Timestamp|
13+
|_cq_id|UUID|
14+
|_cq_parent_id|UUID|
15+
|account_id|String|
16+
|region|String|
17+
|arn (PK)|String|
18+
|administrative_actions|JSON|
19+
|creation_time|Timestamp|
20+
|lifecycle|String|
21+
|lifecycle_transition_reason|JSON|
22+
|name|String|
23+
|snapshot_id|String|
24+
|tags|JSON|
25+
|volume_id|String|

‎plugins/source/aws/docs/tables/aws_lightsail_instance_port_states.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Table: aws_lightsail_instance_port_states
2+
3+
https://docs.aws.amazon.com/lightsail/2016-11-28/api-reference/API_InstancePortState.html
4+
5+
The primary key for this table is **_cq_id**.
6+
7+
## Relations
8+
This table depends on [aws_lightsail_instances](aws_lightsail_instances.md).
9+
10+
## Columns
11+
| Name | Type |
12+
| ------------- | ------------- |
13+
|_cq_source_name|String|
14+
|_cq_sync_time|Timestamp|
15+
|_cq_id (PK)|UUID|
16+
|_cq_parent_id|UUID|
17+
|account_id|String|
18+
|region|String|
19+
|instance_arn|String|
20+
|cidr_list_aliases|StringArray|
21+
|cidrs|StringArray|
22+
|from_port|Int|
23+
|ipv6_cidrs|StringArray|
24+
|protocol|String|
25+
|state|String|
26+
|to_port|Int|

‎plugins/source/aws/docs/tables/aws_lightsail_instances.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Table: aws_lightsail_instances
2+
3+
https://docs.aws.amazon.com/lightsail/2016-11-28/api-reference/API_Instance.html
4+
5+
The primary key for this table is **arn**.
6+
7+
## Relations
8+
The following tables depend on aws_lightsail_instances:
9+
- [aws_lightsail_instance_port_states](aws_lightsail_instance_port_states.md)
10+
11+
## Columns
12+
| Name | Type |
13+
| ------------- | ------------- |
14+
|_cq_source_name|String|
15+
|_cq_sync_time|Timestamp|
16+
|_cq_id|UUID|
17+
|_cq_parent_id|UUID|
18+
|account_id|String|
19+
|region|String|
20+
|access_details|JSON|
21+
|arn (PK)|String|
22+
|add_ons|JSON|
23+
|blueprint_id|String|
24+
|blueprint_name|String|
25+
|bundle_id|String|
26+
|created_at|Timestamp|
27+
|hardware|JSON|
28+
|ip_address_type|String|
29+
|ipv6_addresses|StringArray|
30+
|is_static_ip|Bool|
31+
|location|JSON|
32+
|metadata_options|JSON|
33+
|name|String|
34+
|networking|JSON|
35+
|private_ip_address|String|
36+
|public_ip_address|String|
37+
|resource_type|String|
38+
|ssh_key_name|String|
39+
|state|JSON|
40+
|support_code|String|
41+
|tags|JSON|
42+
|username|String|

0 commit comments

Comments
 (0)