Code relying on AWS IMDSv1 blocking cloud env hardening :sweat: · Issue #1314 · googleapis/google-auth-library-python

Is your feature request related to a problem? Please describe.

AWS IMDSv2 (metadata url - 169.254.169.254) seems to be supported in google/auth/aws.py but some files like google/auth/compute_engine/_metadata.py still rely on IMDSv1, blocking aws cloud hardening.

V1 Legacy code:

google-auth-library-python/google/auth/compute_engine/_metadata.py

Lines 49 to 54 in 9c87ad0

    
           _METADATA_IP_ROOT = "http://{}".format( 
        
               os.getenv(environment_vars.GCE_METADATA_IP, "169.254.169.254") 
        
           ) 
        
           _METADATA_FLAVOR_HEADER = "metadata-flavor" 
        
           _METADATA_FLAVOR_VALUE = "Google" 
        
           _METADATA_HEADERS = {_METADATA_FLAVOR_HEADER: _METADATA_FLAVOR_VALUE}

google-auth-library-python/google/auth/compute_engine/_metadata.py

Lines 153 to 241 in 9c87ad0

    
           def get( 
        
               request, 
        
               path, 
        
               root=_METADATA_ROOT, 
        
               params=None, 
        
               recursive=False, 
        
               retry_count=5, 
        
               headers=None, 
        
           ): 
        
               """Fetch a resource from the metadata server. 
        
               Args: 
        
                   request (google.auth.transport.Request): A callable used to make 
        
                       HTTP requests. 
        
                   path (str): The resource to retrieve. For example, 
        
                       ``'instance/service-accounts/default'``. 
        
                   root (str): The full path to the metadata server root. 
        
                   params (Optional[Mapping[str, str]]): A mapping of query parameter 
        
                       keys to values. 
        
                   recursive (bool): Whether to do a recursive query of metadata. See 
        
                       https://cloud.google.com/compute/docs/metadata#aggcontents for more 
        
                       details. 
        
                   retry_count (int): How many times to attempt connecting to metadata 
        
                       server using above timeout. 
        
                   headers (Optional[Mapping[str, str]]): Headers for the request. 
        
               Returns: 
        
                   Union[Mapping, str]: If the metadata server returns JSON, a mapping of 
        
                       the decoded JSON is return. Otherwise, the response content is 
        
                       returned as a string. 
        
               Raises: 
        
                   google.auth.exceptions.TransportError: if an error occurred while 
        
                       retrieving metadata. 
        
               """ 
        
               base_url = urlparse.urljoin(root, path) 
        
               query_params = {} if params is None else params 
        
               headers_to_use = _METADATA_HEADERS.copy() 
        
               if headers: 
        
                   headers_to_use.update(headers) 
        
               if recursive: 
        
                   query_params["recursive"] = "true" 
        
               url = _helpers.update_query(base_url, query_params) 
        
               retries = 0 
        
               while retries < retry_count: 
        
                   try: 
        
                       response = request(url=url, method="GET", headers=headers_to_use) 
        
                       break 
        
                   except exceptions.TransportError as e: 
        
                       _LOGGER.warning( 
        
                           "Compute Engine Metadata server unavailable on " 
        
                           "attempt %s of %s. Reason: %s", 
        
                           retries + 1, 
        
                           retry_count, 
        
                           e, 
        
                       ) 
        
                       retries += 1 
        
               else: 
        
                   raise exceptions.TransportError( 
        
                       "Failed to retrieve {} from the Google Compute Engine " 
        
                       "metadata service. Compute Engine Metadata server unavailable".format(url) 
        
                   ) 
        
               if response.status == http_client.OK: 
        
                   content = _helpers.from_bytes(response.data) 
        
                   if response.headers["content-type"] == "application/json": 
        
                       try: 
        
                           return json.loads(content) 
        
                       except ValueError as caught_exc: 
        
                           new_exc = exceptions.TransportError( 
        
                               "Received invalid JSON from the Google Compute Engine " 
        
                               "metadata service: {:.20}".format(content) 
        
                           ) 
        
                           six.raise_from(new_exc, caught_exc) 
        
                   else: 
        
                       return content 
        
               else: 
        
                   raise exceptions.TransportError( 
        
                       "Failed to retrieve {} from the Google Compute Engine " 
        
                       "metadata service. Status: {} Response:\n{}".format( 
        
                           url, response.status, response.data 
        
                       ), 
        
                       response, 
        
                   )

V2 Supported code:

google-auth-library-python/google/auth/aws.py

Lines 418 to 470 in 9c87ad0

    
               def retrieve_subject_token(self, request): 
        
                   """Retrieves the subject token using the credential_source object. 
        
                   The subject token is a serialized `AWS GetCallerIdentity signed request`_. 
        
                   The logic is summarized as: 
        
                   Retrieve the AWS region from the AWS_REGION or AWS_DEFAULT_REGION 
        
                   environment variable or from the AWS metadata server availability-zone 
        
                   if not found in the environment variable. 
        
                   Check AWS credentials in environment variables. If not found, retrieve 
        
                   from the AWS metadata server security-credentials endpoint. 
        
                   When retrieving AWS credentials from the metadata server 
        
                   security-credentials endpoint, the AWS role needs to be determined by 
        
                   calling the security-credentials endpoint without any argument. Then the 
        
                   credentials can be retrieved via: security-credentials/role_name 
        
                   Generate the signed request to AWS STS GetCallerIdentity action. 
        
                   Inject x-goog-cloud-target-resource into header and serialize the 
        
                   signed request. This will be the subject-token to pass to GCP STS. 
        
                   .. _AWS GetCallerIdentity signed request: 
        
                       https://cloud.google.com/iam/docs/access-resources-aws#exchange-token 
        
                   Args: 
        
                       request (google.auth.transport.Request): A callable used to make 
        
                           HTTP requests. 
        
                   Returns: 
        
                       str: The retrieved subject token. 
        
                   """ 
        
                   # Fetch the session token required to make meta data endpoint calls to aws. 
        
                   if ( 
        
                       request is not None 
        
                       and self._imdsv2_session_token_url is not None 
        
                       and self._should_use_metadata_server() 
        
                   ): 
        
                       headers = {"X-aws-ec2-metadata-token-ttl-seconds": "300"} 
        
                       imdsv2_session_token_response = request( 
        
                           url=self._imdsv2_session_token_url, method="PUT", headers=headers 
        
                       ) 
        
                       if imdsv2_session_token_response.status != 200: 
        
                           raise exceptions.RefreshError( 
        
                               "Unable to retrieve AWS Session Token", 
        
                               imdsv2_session_token_response.data, 
        
                           ) 
        
                       imdsv2_session_token = imdsv2_session_token_response.data 
        
                   else: 
        
                       imdsv2_session_token = None

Describe the solution you'd like

A full support of IMDSv2 as default metadata fetching service, follwing: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-v2-how-it-works.html

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Code relying on AWS IMDSv1 blocking cloud env hardening 😓 #1314

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	_METADATA_IP_ROOT = "http://{}".format(
	os.getenv(environment_vars.GCE_METADATA_IP, "169.254.169.254")
	)
	_METADATA_FLAVOR_HEADER = "metadata-flavor"
	_METADATA_FLAVOR_VALUE = "Google"
	_METADATA_HEADERS = {_METADATA_FLAVOR_HEADER: _METADATA_FLAVOR_VALUE}

	def get(
	request,
	path,
	root=_METADATA_ROOT,
	params=None,
	recursive=False,
	retry_count=5,
	headers=None,
	):
	"""Fetch a resource from the metadata server.

	Args:
	request (google.auth.transport.Request): A callable used to make
	HTTP requests.
	path (str): The resource to retrieve. For example,
	``'instance/service-accounts/default'``.
	root (str): The full path to the metadata server root.
	params (Optional[Mapping[str, str]]): A mapping of query parameter
	keys to values.
	recursive (bool): Whether to do a recursive query of metadata. See
	https://cloud.google.com/compute/docs/metadata#aggcontents for more
	details.
	retry_count (int): How many times to attempt connecting to metadata
	server using above timeout.
	headers (Optional[Mapping[str, str]]): Headers for the request.

	Returns:
	Union[Mapping, str]: If the metadata server returns JSON, a mapping of
	the decoded JSON is return. Otherwise, the response content is
	returned as a string.

	Raises:
	google.auth.exceptions.TransportError: if an error occurred while
	retrieving metadata.
	"""
	base_url = urlparse.urljoin(root, path)
	query_params = {} if params is None else params

	headers_to_use = _METADATA_HEADERS.copy()
	if headers:
	headers_to_use.update(headers)

	if recursive:
	query_params["recursive"] = "true"

	url = _helpers.update_query(base_url, query_params)

	retries = 0
	while retries < retry_count:
	try:
	response = request(url=url, method="GET", headers=headers_to_use)
	break

	except exceptions.TransportError as e:
	_LOGGER.warning(
	"Compute Engine Metadata server unavailable on "
	"attempt %s of %s. Reason: %s",
	retries + 1,
	retry_count,
	e,
	)
	retries += 1
	else:
	raise exceptions.TransportError(
	"Failed to retrieve {} from the Google Compute Engine "
	"metadata service. Compute Engine Metadata server unavailable".format(url)
	)

	if response.status == http_client.OK:
	content = _helpers.from_bytes(response.data)
	if response.headers["content-type"] == "application/json":
	try:
	return json.loads(content)
	except ValueError as caught_exc:
	new_exc = exceptions.TransportError(
	"Received invalid JSON from the Google Compute Engine "
	"metadata service: {:.20}".format(content)
	)
	six.raise_from(new_exc, caught_exc)
	else:
	return content
	else:
	raise exceptions.TransportError(
	"Failed to retrieve {} from the Google Compute Engine "
	"metadata service. Status: {} Response:\n{}".format(
	url, response.status, response.data
	),
	response,
	)

	def retrieve_subject_token(self, request):
	"""Retrieves the subject token using the credential_source object.
	The subject token is a serialized `AWS GetCallerIdentity signed request`_.

	The logic is summarized as:

	Retrieve the AWS region from the AWS_REGION or AWS_DEFAULT_REGION
	environment variable or from the AWS metadata server availability-zone
	if not found in the environment variable.

	Check AWS credentials in environment variables. If not found, retrieve
	from the AWS metadata server security-credentials endpoint.

	When retrieving AWS credentials from the metadata server
	security-credentials endpoint, the AWS role needs to be determined by
	calling the security-credentials endpoint without any argument. Then the
	credentials can be retrieved via: security-credentials/role_name

	Generate the signed request to AWS STS GetCallerIdentity action.

	Inject x-goog-cloud-target-resource into header and serialize the
	signed request. This will be the subject-token to pass to GCP STS.

	.. _AWS GetCallerIdentity signed request:
	https://cloud.google.com/iam/docs/access-resources-aws#exchange-token

	Args:
	request (google.auth.transport.Request): A callable used to make
	HTTP requests.
	Returns:
	str: The retrieved subject token.
	"""
	# Fetch the session token required to make meta data endpoint calls to aws.
	if (
	request is not None
	and self._imdsv2_session_token_url is not None
	and self._should_use_metadata_server()
	):
	headers = {"X-aws-ec2-metadata-token-ttl-seconds": "300"}

	imdsv2_session_token_response = request(
	url=self._imdsv2_session_token_url, method="PUT", headers=headers
	)

	if imdsv2_session_token_response.status != 200:
	raise exceptions.RefreshError(
	"Unable to retrieve AWS Session Token",
	imdsv2_session_token_response.data,
	)

	imdsv2_session_token = imdsv2_session_token_response.data
	else:
	imdsv2_session_token = None

Code relying on AWS IMDSv1 blocking cloud env hardening 😓 #1314

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions