This repository was archived by the owner on Aug 16, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 39
This repository was archived by the owner on Aug 16, 2022. It is now read-only.
Cloudquery is silently missing resources #955
Copy link
Copy link
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
When I run cloudquery fetch, I expect it to fetch all resources it can.
For instance, I see in the logs
{"level":"info","instance_id":"37cccfe8-c7d4-4e99-b6bf-67af116f01d9","@module":"aws","Region":"us-east-1","account_id":"2265xxxxxxxx","client_id":"aws_ec2_security_groups:2265xxxxxxxx:us-east-1","count":0,"table":"aws_ec2_security_groups","timestamp":"2022-05-26T09:54:02.352-0500","time":"2022-05-26T09:54:02-05:00","message":"fetched successfully"}
However, this account has 92 security groups in that region.
The DB, accordingly, does not contain these groups.
I see this across multiple accounts and regions. I imagine that it also exists for different resources
Expected Behavior
Cloudquery should fetch all resources or show errors if it cannot
Steps to Reproduce
// Configuration AutoGenerated by CloudQuery CLI
cloudquery {
plugin_directory = "./cq/providers"
policy_directory = "./cq/policies"
provider "aws" {
version = "latest"
}
connection {
username = "postgres"
password = "pass"
host = "localhost"
port = 5432
database = "postgres"
sslmode = "disable"
}
}
// All Provider Configurations
provider "aws" {
configuration {
org {
member_role_name = "CloudQueryFetchRole"
admin_account "admin" {
local_profile = "org"
}
}
}
// list of resources to fetch
resources = [
// just the default resources, but I'm showing that we definitely have security groups
"ec2.security_groups",
// ...
]
}
// Module Configurations
modules {
// drift configuration block
drift "drift-example" {
// state block defines from where to access the state
terraform {
// backend: "local" or "s3"
backend = "local"
// local backend options
// files: list of tfstate files
files = ["/path/to.tfstate"]
}
}
}
Then (once the DB is up and running), just run cloudquery fetch
Possible Solution
No response
Provider and CloudQuery version
CQ: 0.24.0, AWS: 0.12.0
Additional Context
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working