Azure all the tables are not getting resources. #5683
Description
Describe the Bug
In Azure Plugin only 70 out of 225 tables are getting data from cloud. In cloudquery logs only 70 services are getting there resolver
client started.
Expected Behavior
All the services should start their resolver client and should fetch resources from the cloud .
CloudQuery Version
I am using cloudquery version 2.0.21
Debug Output
Only these tables are starting their resolver clients
INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=1 table=azure_subscription_tenants
2022-12-15T05:16:05Z ERR table resolver finished with error error="GET https://management.azure.com/global/providers/Microsoft.Authorization/roleDefinitions\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 Not Found\nERROR CODE: MissingSubscription\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"MissingSubscription\",\n \"message\": \"The request did not have a subscription or a valid tenant level resource provider.\"\n }\n}\n--------------------------------------------------------------------------------\n" client=xxxx-xxxx-xxxx-xxxx module=azure-src table=azure_authorization_role_definitions
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=1 module=azure-src resources=0 table=azure_authorization_role_definitions
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_public_ip_addresses
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_public_ip_prefixes
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_virtual_wans
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_keyvault_keyvault
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_application_security_groups
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_express_route_gateways
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_advisor_recommendations
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_load_balancers
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_vpn_gateways
2022-12-15T05:16:05Z ERR table resolver finished with error error="POST https://management.azure.com/providers/Microsoft.Portal/listTenantConfigurationViolations\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 Forbidden\nERROR CODE: AuthorizationFailed\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"AuthorizationFailed\",\n \"message\": \"The client 'd269da12-40cf-4bad-bb5f-c410a00f45d4' with object id 'd269da12-40cf-4bad-bb5f-c410a00f45d4' does not have authorization to perform action 'Microsoft.Portal/listTenantConfigurationViolations/action' over scope '/providers/Microsoft.Portal' or the scope is invalid. If access was recently granted, please refresh your credentials.\"\n }\n}\n--------------------------------------------------------------------------------\n" client=xxxx-xxxx-xxxx-xxxx module=azure-src table=azure_portal_list_tenant_configuration_violations
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=1 module=azure-src resources=0 table=azure_portal_list_tenant_configuration_violations
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_policy_assignments
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_portal_tenant_configurations
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_security_groups
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_azure_firewalls
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=9 table=azure_authorization_role_assignments
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_dnsresolver_dns_resolvers
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_dns_zones
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_dnsresolver_dns_forwarding_rulesets
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=5 table=azure_resources_resources
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=2 table=azure_subscription_subscriptions
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_express_route_circuits
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_privatedns_private_zones
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=9 table=azure_policy_data_policy_manifests
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_policy_exemptions
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_ddos_protection_plans
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_application_gateways
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_web_application_firewall_policies
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_interfaces
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=8 table=azure_network_azure_firewall_fqdn_tags
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_route_filters
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_dscp_configuration
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_ip_allocations
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_virtual_appliances
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_virtual_network_taps
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_profiles
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_vpn_server_configurations
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_virtual_hubs
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_bastion_hosts
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_virtual_routers
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_nat_gateways
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_firewall_policies
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_security_partner_providers
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_express_route_ports
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_service_endpoint_policies
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_ip_groups
2022-12-15T05:16:05Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_route_tables
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_billing_enrollment_accounts
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=88 table=azure_network_express_route_ports_locations
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_costmanagement_views
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_advisor_suppressions
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_vpn_sites
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=1 table=azure_authorization_classic_administrators
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=1 table=azure_storage_accounts
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=4 table=azure_billing_periods
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=261 table=azure_authorization_provider_operations_metadata
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_frontdoor_front_doors
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_custom_ip_prefixes
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=7 table=azure_frontdoor_managed_rule_sets
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=2 table=azure_network_watchers
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=2 table=azure_network_virtual_networks
2022-12-15T05:16:06Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=160 table=azure_network_express_route_service_providers
2022-12-15T05:16:07Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=7 table=azure_advisor_recommendation_metadata
2022-12-15T05:16:07Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_billing_accounts
2022-12-15T05:16:07Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_frontdoor_network_experiment_profiles
2022-12-15T05:16:07Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=84 table=azure_policy_set_definitions
2022-12-15T05:16:07Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=124 table=azure_network_bgp_service_communities
2022-12-15T05:16:08Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=0 table=azure_network_subscription_network_manager_connections
2022-12-15T05:16:09Z INF fetch table finished client=xxxx-xxxx-xxxx-xxxx errors=0 module=azure-src resources=2721 table=azure_policy_definitions
2022-12-15T05:16:09Z INF sync finished duration=3981.339436 errors=2 module=azure-src panics=0 resources=3496
Steps to Reproduce
step1. use source as azure(plugin version=v2.1.1) and destination as postgres(version=1.7.14).
step2. run cloudquery sync command