feat: Move AWS to use built-in transformations (no codegen) by yevgenypats · Pull Request #6337 · cloudquery/cloudquery

yevgenypats · 2023-01-04T12:57:33Z

Similar to #6276 moving transformation to be part of our SDK cloudquery/plugin-sdk#564

This is related to cloudquery/plugin-sdk#573

Few notes on column changes:

most of changes are just additional columns and this should be fine as we did too much transformation. we want to keep the code simple, focus on extraction and provide and easy way of doing "dumb" one level transformation via reflection. The additional column are because we have in some places a duplicate where we add our own arn so it can be consistent but this is our decision and we still want to be as close as possible to the API so some duplicates is fine (and it makes the code simple and consistent).

breaking changes in aws_docdb_pending_maintenance_actions those apparently never worked and those columns where always empty (they didn't have mock tests so we didn't catch those)

github-actions · 2023-01-04T16:00:08Z

This PR has the following changes to source plugin(s) tables:

Table aws_apigatewayv2_apis: column added with name api_id and type String
Table aws_applicationautoscaling_policies: column added with name policy_arn and type String
Table aws_apprunner_auto_scaling_configurations: column added with name auto_scaling_configuration_arn and type String
Table aws_apprunner_connections: column added with name connection_arn and type String
Table aws_apprunner_custom_domains: column order changed for domain_name
Table aws_apprunner_observability_configurations: column added with name observability_configuration_arn and type String
Table aws_apprunner_services: column added with name service_arn and type String
Table aws_apprunner_vpc_connectors: column added with name vpc_connector_arn and type String
Table aws_apprunner_vpc_ingress_connections: column added with name vpc_ingress_connection_arn and type String
Table aws_autoscaling_group_scaling_policies: column added with name policy_arn and type String
Table aws_autoscaling_groups: column added with name auto_scaling_group_arn and type String
Table aws_autoscaling_launch_configurations: column added with name launch_configuration_arn and type String
Table aws_autoscaling_scheduled_actions: column added with name scheduled_action_arn and type String
Table aws_backup_plans: column added with name backup_plan_arn and type String
Table aws_backup_vault_recovery_points: column added with name recovery_point_arn and type String
Table aws_backup_vaults: column added with name backup_vault_arn and type String
Table aws_cloudformation_stacks: column added with name stack_id and type String
Table aws_cloudtrail_trails: column added with name trail_arn and type String
Table aws_cloudwatchlogs_resource_policies: column order changed for last_updated_time
Table aws_cloudwatchlogs_resource_policies: column order changed for policy_name
Table aws_codepipeline_pipelines: column added with name result_metadata and type JSON
Table aws_codepipeline_pipelines: column order changed for pipeline
Table aws_codepipeline_webhooks: column order changed for last_triggered
Table aws_codepipeline_webhooks: column order changed for tags
Table aws_cognito_identity_pools: column added with name identity_pool_id and type String
Table aws_cognito_identity_pools: column order changed for saml_provider_ar_ns
Table aws_config_config_rules: column added with name config_rule_arn and type String
Table aws_config_conformance_packs: column added with name conformance_pack_arn and type String
Table aws_config_conformance_packs: column added with name conformance_pack_input_parameters and type JSON
Table aws_dax_clusters: column added with name cluster_arn and type String
Table aws_directconnect_connections: column added with name connection_id and type String
Table aws_directconnect_gateways: column added with name direct_connect_gateway_id and type String
Table aws_directconnect_lags: column added with name lag_id and type String
Table aws_directconnect_virtual_gateways: column added with name virtual_gateway_id and type String
Table aws_directconnect_virtual_interfaces: column added with name virtual_interface_id and type String
Table aws_directconnect_virtual_interfaces: column order changed for tags
Table aws_dms_replication_instances: column added with name replication_instance_arn and type String
Table aws_docdb_certificates: column added with name certificate_arn and type String
Table aws_docdb_cluster_parameter_groups: column added with name db_cluster_parameter_group_arn and type String
Table aws_docdb_cluster_snapshots: column added with name db_cluster_snapshot_arn and type String
Table aws_docdb_cluster_snapshots: column order changed for availability_zones
Table aws_docdb_cluster_snapshots: column order changed for cluster_create_time
Table aws_docdb_clusters: column added with name db_cluster_arn and type String
Table aws_docdb_global_clusters: column added with name global_cluster_arn and type String
Table aws_docdb_instances: column added with name db_instance_arn and type String
⚠️ BREAKING CHANGE: Table aws_docdb_pending_maintenance_actions: column removed action from table
⚠️ BREAKING CHANGE: Table aws_docdb_pending_maintenance_actions: column removed auto_applied_after_date from table
⚠️ BREAKING CHANGE: Table aws_docdb_pending_maintenance_actions: column removed current_apply_date from table
⚠️ BREAKING CHANGE: Table aws_docdb_pending_maintenance_actions: column removed description from table
⚠️ BREAKING CHANGE: Table aws_docdb_pending_maintenance_actions: column removed forced_apply_date from table
⚠️ BREAKING CHANGE: Table aws_docdb_pending_maintenance_actions: column removed opt_in_status from table
Table aws_docdb_pending_maintenance_actions: column added with name pending_maintenance_action_details and type JSON
Table aws_docdb_pending_maintenance_actions: column added with name resource_identifier and type String
Table aws_docdb_subnet_groups: column added with name db_subnet_group_arn and type String
Table aws_dynamodb_tables: column added with name table_arn and type String
Table aws_ec2_ebs_snapshots: column order changed for tags
Table aws_ec2_ebs_volumes: column order changed for tags
Table aws_ec2_egress_only_internet_gateways: column order changed for egress_only_internet_gateway_id
Table aws_ec2_egress_only_internet_gateways: column order changed for tags
Table aws_ec2_eips: column order changed for public_ipv4_pool
Table aws_ec2_eips: column order changed for tags
Table aws_ec2_flow_logs: column order changed for tags
Table aws_ec2_hosts: column order changed for state
Table aws_ec2_hosts: column order changed for tags
Table aws_ec2_images: column order changed for tags
Table aws_ec2_internet_gateways: column order changed for owner_id
Table aws_ec2_internet_gateways: column order changed for tags
Table aws_ec2_key_pairs: column order changed for public_key
Table aws_ec2_key_pairs: column order changed for tags
Table aws_ec2_nat_gateways: column order changed for tags
Table aws_ec2_network_acls: column order changed for tags
Table aws_ec2_reserved_instances: column order changed for tags
Table aws_ec2_route_tables: column order changed for tags
Table aws_ec2_security_groups: column order changed for tags
Table aws_ec2_subnets: column order changed for tags
Table aws_ec2_transit_gateway_attachments: column order changed for tags
Table aws_ec2_transit_gateway_multicast_domains: column order changed for tags
Table aws_ec2_transit_gateway_peering_attachments: column order changed for tags
Table aws_ec2_transit_gateway_route_tables: column order changed for tags
Table aws_ec2_transit_gateways: column added with name transit_gateway_arn and type String
Table aws_ec2_transit_gateways: column added with name transit_gateway_id and type String
Table aws_ec2_transit_gateways: column order changed for tags
Table aws_ec2_vpc_endpoint_service_configurations: column order changed for supported_ip_address_types
Table aws_ec2_vpc_endpoint_service_configurations: column order changed for tags
Table aws_ec2_vpc_endpoint_services: column order changed for tags
Table aws_ec2_vpc_endpoints: column order changed for tags
Table aws_ec2_vpc_peering_connections: column order changed for tags
Table aws_ec2_vpcs: column order changed for tags
Table aws_ec2_vpn_gateways: column order changed for tags
Table aws_ecr_registries: column added with name result_metadata and type JSON
Table aws_ecr_registries: column order changed for replication_configuration
Table aws_ecr_registry_policies: column added with name result_metadata and type JSON
Table aws_ecr_registry_policies: column order changed for policy_text
Table aws_ecr_repositories: column added with name repository_arn and type String
Table aws_ecrpublic_repositories: column added with name repository_arn and type String
Table aws_ecs_cluster_services: column added with name service_arn and type String
Table aws_ecs_cluster_services: column order changed for tags
Table aws_ecs_cluster_tasks: column added with name task_arn and type String
Table aws_ecs_clusters: column added with name cluster_arn and type String
Table aws_ecs_clusters: column order changed for status
Table aws_ecs_clusters: column order changed for tags
Table aws_ecs_task_definitions: column added with name task_definition_arn and type String
Table aws_efs_filesystems: column added with name file_system_arn and type String
Table aws_efs_filesystems: column order changed for tags
Table aws_eks_clusters: column order changed for tags
Table aws_elasticache_reserved_cache_nodes: column added with name reservation_arn and type String
Table aws_elasticbeanstalk_application_versions: column added with name application_version_arn and type String
Table aws_elasticbeanstalk_applications: column added with name application_arn and type String
Table aws_elasticbeanstalk_environments: column added with name environment_arn and type String
Table aws_elasticbeanstalk_environments: column added with name environment_id and type String
Table aws_elasticsearch_packages: column added with name package_id and type String
Table aws_elasticsearch_packages: column order changed for id
Table aws_elasticsearch_vpc_endpoints: column added with name vpc_endpoint_id and type String
Table aws_elasticsearch_vpc_endpoints: column order changed for id
Table aws_elbv2_listeners: column added with name listener_arn and type String
Table aws_elbv2_load_balancers: column added with name load_balancer_arn and type String
Table aws_elbv2_target_groups: column added with name target_group_arn and type String
Table aws_emr_clusters: column added with name cluster_arn and type String
Table aws_emr_clusters: column order changed for tags
Table aws_eventbridge_api_destinations: column added with name api_destination_arn and type String
Table aws_eventbridge_connections: column added with name connection_arn and type String
Table aws_firehose_delivery_streams: column added with name delivery_stream_arn and type String
Table aws_fsx_backups: column added with name backup_id and type String
Table aws_fsx_backups: column order changed for tags
Table aws_fsx_data_repository_associations: column added with name resource_arn and type String
Table aws_fsx_data_repository_associations: column order changed for s3
Table aws_fsx_data_repository_associations: column order changed for tags
Table aws_fsx_data_repository_tasks: column added with name resource_arn and type String
Table aws_fsx_data_repository_tasks: column order changed for status
Table aws_fsx_data_repository_tasks: column order changed for tags
Table aws_fsx_file_caches: column added with name resource_arn and type String
Table aws_fsx_file_systems: column added with name resource_arn and type String
Table aws_fsx_snapshots: column added with name resource_arn and type String
Table aws_fsx_snapshots: column order changed for tags
Table aws_fsx_storage_virtual_machines: column added with name resource_arn and type String
Table aws_fsx_storage_virtual_machines: column order changed for tags
Table aws_fsx_volumes: column added with name resource_arn and type String
Table aws_fsx_volumes: column order changed for tags
Table aws_glacier_vault_lock_policies: column added with name creation_date and type String
Table aws_glacier_vault_lock_policies: column added with name expiration_date and type String
Table aws_glacier_vault_lock_policies: column added with name state and type String
Table aws_glacier_vault_lock_policies: column order changed for policy
Table aws_glacier_vaults: column added with name vault_arn and type String
Table aws_glue_registries: column added with name registry_arn and type String
Table aws_glue_registry_schemas: column added with name schema_arn and type String
Table aws_iam_groups: column added with name group_id and type String
Table aws_iam_openid_connect_identity_providers: column order changed for tags
Table aws_iam_policies: column added with name policy_id and type String
Table aws_iam_roles: column added with name role_id and type String
Table aws_iam_roles: column order changed for role_last_used
Table aws_iam_roles: column order changed for tags
Table aws_iam_saml_identity_providers: column added with name saml_metadata_document and type String
Table aws_iam_saml_identity_providers: column added with name tags and type JSON
Table aws_iam_server_certificates: column added with name server_certificate_id and type String
Table aws_iam_users: column added with name user_id and type String
Table aws_iam_users: column order changed for permissions_boundary
Table aws_iam_users: column order changed for tags
Table aws_iam_virtual_mfa_devices: column order changed for tags
Table aws_inspector2_findings: column added with name finding_arn and type String
Table aws_iot_billing_groups: column added with name billing_group_arn and type String
Table aws_iot_ca_certificates: column added with name certificate_arn and type String
Table aws_iot_certificates: column added with name certificate_arn and type String
Table aws_iot_jobs: column added with name job_arn and type String
Table aws_iot_policies: column added with name policy_arn and type String
Table aws_iot_streams: column added with name stream_arn and type String
Table aws_iot_thing_groups: column added with name thing_group_arn and type String
Table aws_iot_thing_types: column added with name thing_type_arn and type String
Table aws_iot_things: column added with name thing_arn and type String
Table aws_iot_topic_rules: column added with name rule_arn and type String
Table aws_kafka_cluster_operations: column added with name operation_arn and type String
Table aws_kafka_clusters: column added with name cluster_arn and type String
Table aws_kafka_nodes: column added with name node_arn and type String
Table aws_kinesis_streams: column added with name stream_arn and type String
Table aws_kms_aliases: column added with name alias_arn and type String
Table aws_lambda_function_aliases: column added with name alias_arn and type String
Table aws_lambda_layer_versions: column added with name layer_version_arn and type String
Table aws_lightsail_buckets: column order changed for tags
Table aws_lightsail_container_services: column order changed for tags
Table aws_lightsail_database_snapshots: column order changed for support_code
Table aws_lightsail_database_snapshots: column order changed for tags
Table aws_lightsail_databases: column order changed for support_code
Table aws_lightsail_databases: column order changed for tags
Table aws_lightsail_disk_snapshots: column order changed for support_code
Table aws_lightsail_disk_snapshots: column order changed for tags
Table aws_lightsail_instance_snapshots: column order changed for support_code
Table aws_lightsail_instance_snapshots: column order changed for tags
Table aws_lightsail_instances: column order changed for tags
Table aws_lightsail_load_balancer_tls_certificates: column order changed for support_code
Table aws_lightsail_load_balancer_tls_certificates: column order changed for tags
Table aws_lightsail_load_balancers: column order changed for tags
Table aws_mq_brokers: column added with name broker_arn and type String
Table aws_neptune_cluster_parameter_groups: column added with name db_cluster_parameter_group_arn and type String
Table aws_neptune_cluster_snapshots: column added with name db_cluster_snapshot_arn and type String
Table aws_neptune_clusters: column added with name db_cluster_arn and type String
Table aws_neptune_db_parameter_groups: column added with name db_parameter_group_arn and type String
Table aws_neptune_event_subscriptions: column added with name event_subscription_arn and type String
Table aws_neptune_global_clusters: column added with name global_cluster_arn and type String
Table aws_neptune_instances: column added with name db_instance_arn and type String
Table aws_neptune_subnet_groups: column added with name db_subnet_group_arn and type String
Table aws_neptune_subnet_groups: column order changed for vpc_id
Table aws_quicksight_group_members: column added with name arn and type String
Table aws_ram_principals: column order changed for id
Table aws_ram_resource_share_associations: column order changed for resource_share_arn
Table aws_ram_resource_share_invitations: column added with name resource_share_invitation_arn and type String
Table aws_ram_resource_share_invitations: column order changed for arn
Table aws_ram_resource_shares: column added with name resource_share_arn and type String
Table aws_ram_resource_shares: column order changed for arn
Table aws_ram_resource_shares: column order changed for status_message
Table aws_ram_resource_shares: column order changed for tags
Table aws_ram_resource_types: column order changed for resource_region_scope
Table aws_ram_resource_types: column order changed for service_name
Table aws_rds_certificates: column added with name certificate_arn and type String
Table aws_rds_cluster_parameter_groups: column added with name db_cluster_parameter_group_arn and type String
Table aws_rds_cluster_snapshots: column added with name db_cluster_snapshot_arn and type String
Table aws_rds_clusters: column added with name db_cluster_arn and type String
Table aws_rds_db_parameter_groups: column added with name db_parameter_group_arn and type String
Table aws_rds_db_security_groups: column added with name db_security_group_arn and type String
Table aws_rds_db_snapshots: column added with name db_snapshot_arn and type String
Table aws_rds_engine_versions: column order changed for tag_list
Table aws_rds_event_subscriptions: column added with name event_subscription_arn and type String
Table aws_rds_instances: column added with name db_instance_arn and type String
Table aws_rds_subnet_groups: column added with name db_subnet_group_arn and type String
Table aws_redshift_event_subscriptions: column order changed for subscription_creation_time
Table aws_redshift_event_subscriptions: column order changed for tags
Table aws_redshift_subnet_groups: column order changed for tags
Table aws_regions: column added with name region_name and type String
Table aws_regions: column order changed for opt_in_status
Table aws_sagemaker_endpoint_configurations: column added with name endpoint_config_arn and type String
Table aws_sagemaker_models: column added with name model_arn and type String
Table aws_sagemaker_notebook_instances: column added with name notebook_instance_arn and type String
Table aws_sagemaker_training_jobs: column added with name training_job_arn and type String
Table aws_secretsmanager_secrets: column order changed for tags
Table aws_servicecatalog_products: column added with name product_arn and type String
Table aws_servicequotas_quotas: column added with name quota_arn and type String
Table aws_ses_configuration_set_event_destinations: column order changed for matching_event_types
Table aws_ses_configuration_sets: column order changed for tags
Table aws_ses_contact_lists: column added with name contact_list_name and type String
Table aws_ses_contact_lists: column order changed for tags
Table aws_ses_identities: column order changed for tags
Table aws_shield_attacks: column added with name attack_id and type String
Table aws_shield_protection_groups: column added with name protection_group_arn and type String
Table aws_shield_protections: column added with name protection_arn and type String
Table aws_shield_subscriptions: column added with name subscription_arn and type String
Table aws_sns_subscriptions: column added with name subscription_arn and type String
Table aws_sqs_queues: column order changed for tags
Table aws_ssm_documents: column order changed for tags
Table aws_stepfunctions_state_machines: column added with name state_machine_arn and type String
Table aws_waf_rule_groups: column added with name metric_name and type String
Table aws_waf_rule_groups: column order changed for name
Table aws_waf_rule_groups: column order changed for rule_group_id
Table aws_waf_rules: column added with name metric_name and type String
Table aws_waf_rules: column added with name predicates and type JSON
Table aws_waf_rules: column order changed for name
Table aws_waf_rules: column order changed for rule_id
Table aws_waf_web_acls: column added with name default_action and type JSON
Table aws_waf_web_acls: column added with name metric_name and type String
Table aws_waf_web_acls: column added with name rules and type JSON
Table aws_waf_web_acls: column added with name web_acl_arn and type String
Table aws_waf_web_acls: column order changed for logging_configuration
Table aws_waf_web_acls: column order changed for web_acl_id
Table aws_wafregional_web_acls: column added with name web_acl_arn and type String
Table aws_wafregional_web_acls: column order changed for name
Table aws_xray_groups: column added with name group_arn and type String

vercel · 2023-01-04T16:02:01Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
cloudquery-web	🔄 Building (Inspect)		Jan 5, 2023 at 2:47PM (UTC)

…g from Z) (#6369) Co-authored-by: Kemal Hadimli <disq@users.noreply.github.com>

plugins/source/aws/resources/services/sns/subscriptions.go

plugins/source/aws/resources/services/sns/topics.go

plugins/source/aws/resources/services/sqs/queues.go

Similar to #6337 on our way to deprecate codegen

hermanschaaf · 2023-01-06T14:31:46Z

Bringing an offline conversation here: I think we should keep the codegen that generates service interfaces for mocks, so those don't need to be written by hand. However, we can now make it part of the make gen-mocks step, so it's not something you're going to need to run often (only when adding a completely new service).

We should also make sure the Contributing doc is up-to-date, because I think it still references codegen.

🤖 I have created a release *beep* *boop* --- ## [9.2.0](plugins-source-aws-v9.1.1...plugins-source-aws-v9.2.0) (2023-01-10) ### Features * **aws:** KMS Key policies ([#6562](#6562)) ([d8571aa](d8571aa)), closes [#6559](#6559) * Move AWS to use built-in transformations (no codegen) ([#6337](#6337)) ([926d278](926d278)) ### Bug Fixes * **aws:** Enable Partial Resolution of Lambda ([#6391](#6391)) ([e96a410](e96a410)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/config to v1.18.8 ([#6393](#6393)) ([ffba44f](ffba44f)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/feature/s3/manager to v1.11.47 ([#6395](#6395)) ([71ec9b9](71ec9b9)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/accessanalyzer to v1.19.0 ([#6396](#6396)) ([f19a34c](f19a34c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/account to v1.8.0 ([#6397](#6397)) ([98a7681](98a7681)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/acm to v1.17.0 ([#6398](#6398)) ([f59f6bd](f59f6bd)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/amp to v1.16.0 ([#6399](#6399)) ([128ae76](128ae76)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apigateway to v1.16.0 ([#6400](#6400)) ([4570f15](4570f15)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apigatewayv2 to v1.13.0 ([#6401](#6401)) ([ca3b156](ca3b156)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/applicationautoscaling to v1.16.0 ([#6360](#6360)) ([afa651e](afa651e)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/applicationautoscaling to v1.17.0 ([#6402](#6402)) ([5c239ed](5c239ed)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apprunner to v1.16.0 ([#6403](#6403)) ([a312c4b](a312c4b)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/appstream to v1.19.0 ([#6404](#6404)) ([1cd08e1](1cd08e1)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/appsync to v1.18.0 ([#6405](#6405)) ([76d4df5](76d4df5)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/athena to v1.22.0 ([#6406](#6406)) ([815a0f9](815a0f9)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/autoscaling to v1.26.0 ([#6407](#6407)) ([26e995c](26e995c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/backup to v1.19.0 ([#6408](#6408)) ([01accc8](01accc8)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudformation to v1.25.0 ([#6409](#6409)) ([89ffac6](89ffac6)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudfront to v1.24.0 ([#6410](#6410)) ([74a96fb](74a96fb)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudhsmv2 to v1.14.0 ([#6411](#6411)) ([d12e25c](d12e25c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudtrail to v1.22.0 ([#6412](#6412)) ([c1765b8](c1765b8)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatch to v1.24.0 ([#6413](#6413)) ([c6324f3](c6324f3)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to v1.17.4 ([#6359](#6359)) ([8ca9085](8ca9085)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to v1.18.0 ([#6414](#6414)) ([35b62ac](35b62ac)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/codebuild to v1.20.0 ([#6444](#6444)) ([b91ca03](b91ca03)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/codepipeline to v1.14.0 ([#6445](#6445)) ([bb7d3f2](bb7d3f2)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cognitoidentity to v1.15.0 ([#6446](#6446)) ([c93c305](c93c305)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cognitoidentityprovider to v1.22.0 ([#6447](#6447)) ([4ffbfd5](4ffbfd5)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/configservice to v1.29.0 ([#6448](#6448)) ([2433cbe](2433cbe)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/databasemigrationservice to v1.23.0 ([#6449](#6449)) ([9c2bb7c](9c2bb7c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/dax to v1.12.0 ([#6450](#6450)) ([da537c6](da537c6)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/directconnect to v1.18.0 ([#6451](#6451)) ([89e34fe](89e34fe)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/docdb to v1.20.0 ([#6452](#6452)) ([472536a](472536a)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/dynamodb to v1.18.0 ([#6453](#6453)) ([561db49](561db49)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecr to v1.18.0 ([#6454](#6454)) ([ecdbe98](ecdbe98)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecrpublic to v1.14.0 ([#6455](#6455)) ([7984da9](7984da9)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecs to v1.23.0 ([#6456](#6456)) ([94b8b4c](94b8b4c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/efs to v1.19.0 ([#6457](#6457)) ([a56841a](a56841a)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/eks to v1.27.0 ([#6458](#6458)) ([4a74a54](4a74a54)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticache to v1.26.0 ([#6459](#6459)) ([5c70e45](5c70e45)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticbeanstalk to v1.15.0 ([#6460](#6460)) ([3333cfb](3333cfb)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticloadbalancing to v1.15.0 ([#6461](#6461)) ([d72d213](d72d213)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to v1.19.0 ([#6462](#6462)) ([57ba804](57ba804)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticsearchservice to v1.18.0 ([#6463](#6463)) ([e282e84](e282e84)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elastictranscoder to v1.14.0 ([#6464](#6464)) ([163a581](163a581)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/emr to v1.22.0 ([#6465](#6465)) ([f149f08](f149f08)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/eventbridge to v1.17.0 ([#6473](#6473)) ([188848f](188848f)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/firehose to v1.16.0 ([#6474](#6474)) ([de60adb](de60adb)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/frauddetector to v1.21.0 ([#6475](#6475)) ([19b154a](19b154a)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/fsx to v1.28.0 ([#6476](#6476)) ([7b39554](7b39554)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/glacier to v1.14.0 ([#6477](#6477)) ([8d0887f](8d0887f)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/glue to v1.39.0 ([#6478](#6478)) ([4001267](4001267)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/guardduty to v1.17.0 ([#6479](#6479)) ([651dfe4](651dfe4)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/iam to v1.19.0 ([#6480](#6480)) ([30cf79d](30cf79d)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/identitystore to v1.16.0 ([#6481](#6481)) ([c7dd4a6](c7dd4a6)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/inspector to v1.13.0 ([#6482](#6482)) ([e628a51](e628a51)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/inspector2 to v1.11.0 ([#6483](#6483)) ([fe8fd70](fe8fd70)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/iot to v1.33.0 ([#6484](#6484)) ([d66ca25](d66ca25)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/kafka to v1.19.0 ([#6485](#6485)) ([acb2683](acb2683)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/kinesis to v1.17.0 ([#6486](#6486)) ([31b551f](31b551f)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/kms to v1.20.0 ([#6487](#6487)) ([5a8b81d](5a8b81d)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/lambda to v1.27.0 ([#6488](#6488)) ([de83eda](de83eda)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/lightsail to v1.25.0 ([#6489](#6489)) ([99ce478](99ce478)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/mq to v1.14.0 ([#6490](#6490)) ([d5b465c](d5b465c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/mwaa to v1.14.0 ([#6491](#6491)) ([c565b0a](c565b0a)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/neptune to v1.19.0 ([#6492](#6492)) ([d373abd](d373abd)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/organizations to v1.18.0 ([#6495](#6495)) ([bbb2457](bbb2457)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/qldb to v1.15.0 ([#6496](#6496)) ([5f01dcf](5f01dcf)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/quicksight to v1.29.0 ([#6497](#6497)) ([6c6788c](6c6788c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ram to v1.17.0 ([#6498](#6498)) ([306bcc5](306bcc5)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/rds to v1.39.0 ([#6499](#6499)) ([ed4cd10](ed4cd10)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/redshift to v1.27.0 ([#6500](#6500)) ([4c82e8b](4c82e8b)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/resourcegroups to v1.13.0 ([#6501](#6501)) ([9efd121](9efd121)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/route53 to v1.26.0 ([#6502](#6502)) ([135dc6b](135dc6b)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/route53domains to v1.14.0 ([#6503](#6503)) ([ba2aa9e](ba2aa9e)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/s3control to v1.29.0 ([#6505](#6505)) ([47fa203](47fa203)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/sagemaker to v1.62.0 ([#6506](#6506)) ([57207cf](57207cf)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/scheduler to v1.1.0 ([#6507](#6507)) ([a56a3e0](a56a3e0)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/secretsmanager to v1.18.0 ([#6508](#6508)) ([83f6116](83f6116)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/servicecatalog to v1.16.0 ([#6509](#6509)) ([1e69f8d](1e69f8d)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/servicecatalogappregistry to v1.16.0 ([#6510](#6510)) ([6a7f928](6a7f928)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/servicequotas to v1.14.0 ([#6511](#6511)) ([d12e3b8](d12e3b8)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ses to v1.15.0 ([#6512](#6512)) ([4f8863c](4f8863c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/sesv2 to v1.16.0 ([#6513](#6513)) ([a4987ae](a4987ae)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/sfn to v1.17.0 ([#6514](#6514)) ([c8e7d9c](c8e7d9c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ssm to v1.34.0 ([#6361](#6361)) ([2413dba](2413dba)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.18.0 ([#6339](#6339)) ([158365a](158365a)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.19.0 ([#6363](#6363)) ([ae6967c](ae6967c)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.20.0 ([#6376](#6376)) ([d6187ec](d6187ec)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.21.0 ([#6382](#6382)) ([5baea40](5baea40)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.22.0 ([#6516](#6516)) ([b7e4e73](b7e4e73)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.23.0 ([#6522](#6522)) ([ce24f1d](ce24f1d)) * Update endpoints ([#6327](#6327)) ([11d6973](11d6973)) * Update endpoints ([#6440](#6440)) ([499f421](499f421)) * Update endpoints ([#6558](#6558)) ([45c4753](45c4753)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

cq-bot added the aws label Jan 4, 2023

yevgenypats changed the title ~~Feat/aws codegenless~~ feat: Move AWS to use built-in transformations (no codegen) Jan 4, 2023

yevgenypats marked this pull request as ready for review January 4, 2023 22:10

yevgenypats requested a review from hermanschaaf as a code owner January 4, 2023 22:10

yevgenypats requested a review from erezrokah January 4, 2023 22:14

yevgenypats force-pushed the feat/aws_codegenless branch from 9b52380 to 9de1841 Compare January 5, 2023 09:06

yevgenypats requested a review from disq January 5, 2023 09:06

yevgenypats added 21 commits January 5, 2023 22:49

feat(aws): Remove codegen, use built-in transformers

131102c

done till cloudwatchlogs

92fe7d8

cognito

8f4fda8

dynamodb

7e57535

fix makefile

d6d9061

breaking changes fixes

8c36354

more fixes for breaking changes

f85762d

fix breaking changes

df17cdf

fmt

a24ff5f

update md

35525ab

add mock tests for docdb

98b52a4

fix gomod

0dd8f66

ec2

769bf21

fmt

ff6fbc2

ecs

4ab9b22

elasticache

a010011

elasticbeanstalk

14e1b3f

elasticsearch

1ad8d22

Add elbv2

1ce620a

fmt

9bfc0be

emr

826c2b5

yevgenypats and others added 6 commits January 5, 2023 22:50

add eventbridge

8c2128d

firehose

bcdf7c5

frauddetector

bf97aaa

feat(aws): Remove codegen, use built-in transformers (Part 3: startin…

48b2df5

…g from Z) (#6369) Co-authored-by: Kemal Hadimli <disq@users.noreply.github.com>

fix columns

885b49c

minot tweaks

f8e0cb5

yevgenypats force-pushed the feat/aws_codegenless branch from 72d9114 to f8e0cb5 Compare January 5, 2023 20:51

disq reviewed Jan 5, 2023

View reviewed changes

plugins/source/aws/resources/services/sns/subscriptions.go Show resolved Hide resolved

disq reviewed Jan 5, 2023

View reviewed changes

plugins/source/aws/resources/services/sns/topics.go Show resolved Hide resolved

disq reviewed Jan 5, 2023

View reviewed changes

plugins/source/aws/resources/services/sqs/queues.go Show resolved Hide resolved

yevgenypats mentioned this pull request Jan 6, 2023

feat(cloudflare): Remove codegen, move to transformation #6420

Merged

hermanschaaf approved these changes Jan 6, 2023

View reviewed changes

Merge branch 'main' into feat/aws_codegenless

fac472a

kodiakhq bot pushed a commit that referenced this pull request Jan 6, 2023

feat(cloudflare): Remove codegen, move to transformation (#6420)

881f5f3

Similar to #6337 on our way to deprecate codegen

Merge branch 'main' into feat/aws_codegenless

a281b5a

vercel bot deployed to Preview January 9, 2023 09:06 View deployment

yevgenypats added 2 commits January 9, 2023 11:13

trigger workflows

487e838

fmt

6fe6cac

yevgenypats merged commit 926d278 into main Jan 9, 2023

yevgenypats deleted the feat/aws_codegenless branch January 9, 2023 09:42

cq-bot mentioned this pull request Jan 9, 2023

chore(main): Release plugins-source-aws v9.2.0 #6328

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Move AWS to use built-in transformations (no codegen)#6337

feat: Move AWS to use built-in transformations (no codegen)#6337
yevgenypats merged 31 commits intomainfrom
feat/aws_codegenless

yevgenypats commented Jan 4, 2023 •

edited by hermanschaaf

Loading

Uh oh!

github-actions bot commented Jan 4, 2023 •

edited

Loading

Uh oh!

vercel bot commented Jan 4, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hermanschaaf commented Jan 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

yevgenypats commented Jan 4, 2023 • edited by hermanschaaf Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This PR has the following changes to source plugin(s) tables:

Uh oh!

vercel bot commented Jan 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hermanschaaf commented Jan 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yevgenypats commented Jan 4, 2023 •

edited by hermanschaaf

Loading

github-actions bot commented Jan 4, 2023 •

edited

Loading

vercel bot commented Jan 4, 2023 •

edited

Loading