Skip to content

What should Cloudquery plugins consider as "errors" #14652

@AshCorr

Description

@AshCorr

False positive for Errors

We've noticed a few tables from the AWS source plugin throwing seemingly false positives for errors in perfectly normal situations:

  1. aws_s3_bucket_object_lock_configurations

    An error will be thrown by every S3 bucket that doesn't have Object Lock enabled as the AWS API will return a 404 response code when requesting bucket lock information.

  2. aws_lambda_functions

    Any Lambda that uses a Image runtime will throw 2 errors when it tries to call the GetRuntimeManagementConfig and GetFunctionCodeSigningConfig AWS APIs, as neither API support image runtime Lambdas.

  3. aws_ecr_repository_lifecycle_policies

    An error will be thrown by every ECR Repository that doesn't have a lifecycle policy as the AWS API will return a 404 response code when requesting lifecycle policy information.

this is not an exclusive list

In my opinion, it feels like these should be considered either "warnings" or "info", not "errors", as they're not caused by any underlying api outage/cloudquery misconfiguration/rate limit/etc but instead caused by perfectly normal AWS Resources.

Impact

Ideally we'd be able to scan our logs for errors and alert us if anything comes up that needs our attention, for example if an API outage caused gaps in our data.

Theres a "table sync finished" log message that gets posted after every table has finished syncing which has a "errors" field indicating the total number of errors encountered. But with how many false positives there are it becomes un-usable

image

We're happy to look at fixing some of these errors ourselves, but wanted to understand what the Cloudquery team considers an "error" to be before we do any work! Maybe some suggestions to plugin developers on what should be considered an error?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions