TiKV aws storage sdk don't support retrieve instance metadata via IMDSv2 #16443
Description
Bug Report
What version of TiKV are you using?
v6.5.6-fips
What operating system and CPU are you using?
Steps to reproduce
BR back full database to s3， while the tikv pod has access permission to s3，back still report following error
Error: error happen in store 1 at basic-tikv-2.basic-tikv-peer.tidb-cluster.svc:20160: Io(Custom { kind: Other, error: "failed to put object rusoto error Couldn't find AWS credentials in sources (Couldn't find AWS credentials in environment, credentials file, or IAM role;No (or empty) AWS_ACCESS_KEY_ID in environment;profile not found;Could not get request from environment: Neither environment variable 'AWS_CONTAINER_CREDENTIALS_FULL_URI' nor 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI' is set;EOF while parsing a value at line 1 column 0)." }): [BR:KV:ErrKVStorage]tikv storage occur I/O error
What did you expect?
What did happened?
Check in the pod, found that it only support get meta data via IMDSv2
[ec2-user@ip-172-31-5-204 ~]$ curl -vvv http://169.254.169.254/latest/meta-data/
* Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> GET /latest/meta-data/ HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Content-Length: 0
< Date: Wed, 24 Jan 2024 13:35:24 GMT
< Server: EC2ws
< Connection: close
< Content-Type: text/plain
<
* Closing connection 0
[ec2-user@ip-172-31-5-204 ~]$ Invoke-RestMethod -Headers @{"X-aws-ec2-metadata-token" = $token} -Method GET -Uri http://169.254.169.254/
-bash: Invoke-RestMethod: command not found
[ec2-user@ip-172-31-5-204 ~]$ [string]$token = Invoke-RestMethod -Headers @{"X-aws-ec2-metadata-token-ttl-seconds" = "21600"} -Method PUT -Uri http://169.254.169.254/latest/api/token
-bash: [string]: command not found
[ec2-user@ip-172-31-5-204 ~]$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56 100 56 0 0 56000 0 --:--:-- --:--:-- --:--:-- 56000
* Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> GET /latest/meta-data/ HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.76.1
> Accept: */*
> X-aws-ec2-metadata-token: AQAEAFbt_8QR9s8xZq8qhyVsNnSlOiiEtF906h66mWTOjsIgneLNyg==
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
< Content-Type: text/plain
< Accept-Ranges: none
< Last-Modified: Wed, 24 Jan 2024 12:48:28 GMT
< Content-Length: 325
< Date: Wed, 24 Jan 2024 13:41:14 GMT
< Server: EC2ws
< Connection: close
<
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups
services/
* Closing connection 0
w/a is modifying instance meta data option for IMDSv2 from required to optional
