"Unable to determine availability zone ID" warnings #9142
Description
As the contributor of the new __meta_ec2_availability_zone_id label, I think there are two actions here:
- Highlight somewhere in the release notes that EC2 SD now requires
ec2:DescribeAvailabilityZonesto avoid warnings. - Modify the refresh logic to only describe AZ IDs once per refresh. As it stands, a call will be made for every target, which could be a lot of requests. I'll raise a PR for this.
Aug 02 12:15:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:15:48.890Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2a err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 7315bd5d-1d4b-4e15-bdef-cd89023d1810"
Aug 02 12:15:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:15:48.894Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2c err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: ed619996-3446-42e8-b6f0-17e73c12d119"
Aug 02 12:16:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:16:48.883Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2a err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 68af8117-31a8-4fb8-abdc-969d4deaf229"
Aug 02 12:16:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:16:48.888Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2c err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: b9b172e6-dd17-484e-9b37-dd68cab45af1"
Aug 02 12:16:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:16:48.892Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2a err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 964e59a4-4551-4b84-8ead-f62d86cd5f3e"
Aug 02 12:16:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:16:48.896Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2c err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: c2692349-0392-4204-ba70-508d9c494874"
Aug 02 12:17:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:17:48.880Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2a err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: b9c37a4b-789a-4963-8655-db4671bf8cf4"
Aug 02 12:17:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:17:48.884Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2c err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: efc89d50-fbf9-43a2-9b63-c527dcffccf5"
Aug 02 12:17:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:17:48.889Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2a err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 1d55608a-def0-4757-b401-be726630031d"
Aug 02 12:17:48 ip-10-22-1-4 prometheus[16964]: level=warn ts=2021-08-02T12:17:48.893Z caller=ec2.go:245 component="discovery manager scrape" discovery=ec2 msg="Unable to determine availability zone ID" az=eu-west-2c err="could not describe availability zones: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: b197e921-af02-4320-b950-c98f43c2d542"