How to workaround duplicate metrics from Stackdriver #103
Description
Hello there,
I'm giving it a shot here.
Is there anything we can do other than create a white/blacklist for these trouble maker metrics for the exporter?
For example, this one is from Metric: logging.googleapis.com/log_entry_count, Resource Type: spanner_instance and we see it happen only from US-based projects. From GCP, Metrics explorer, you also see it is not really unique.
* [from Gatherer #2] collected metric "stackdriver_spanner_instance_logging_googleapis_com_byte_count" { label:<name:"instance_config" value:"" > label:<name:"instance_id" value:"spanner-001" > label:<name:"location" value:"us-east1" > label:<name:"log" value:"cloudaudit.googleapis.com/data_access" > label:<name:"project_id" value:"my-project-id" > label:<name:"severity" value:"INFO" > label:<name:"unit" value:"By" > gauge:<value:32656 > timestamp_ms:1594973553075 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_spanner_instance_logging_googleapis_com_byte_count" { label:<name:"instance_config" value:"" > label:<name:"instance_id" value:"spanner-002" > label:<name:"location" value:"us-east1" > label:<name:"log" value:"cloudaudit.googleapis.com/data_access" > label:<name:"project_id" value:"my-project-id" > label:<name:"severity" value:"INFO" > label:<name:"unit" value:"By" > gauge:<value:19902 > timestamp_ms:1594973553075 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_spanner_instance_logging_googleapis_com_byte_count" { label:<name:"instance_config" value:"" > label:<name:"instance_id" value:"spanner-003" > label:<name:"location" value:"us-east1" > label:<name:"log" value:"cloudaudit.googleapis.com/data_access" > label:<name:"project_id" value:"my-project-id" > label:<name:"severity" value:"INFO" > label:<name:"unit" value:"By" > gauge:<value:0 > timestamp_ms:1594973433075 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_spanner_instance_logging_googleapis_com_byte_count" { label:<name:"instance_config" value:"" > label:<name:"instance_id" value:"spanner-004" > label:<name:"location" value:"us-east1" > label:<name:"log" value:"cloudaudit.googleapis.com/data_access" > label:<name:"project_id" value:"my-project-id" > label:<name:"severity" value:"INFO" > label:<name:"unit" value:"By" > gauge:<value:28341 > timestamp_ms:1594973553075 } was collected before with the same name and label values
Cordially,
// Nakarin