Skip to content

(aws-eks): Construct Library custom resources doesn't use proxy properly  #12469

@oleksii-boiko-ua

Description

@oleksii-boiko-ua

I'm trying to test new feature related to provision within the VPC lambda functions related to EKS configuration like ClusterHandler. Lambdas are placed to vpc and it's great. But I got error connecting to EKS api via proxy

Reproduction Steps

self.cluster = aws_eks.Cluster(
            scope=self,
            id='cluster',
            cluster_name="cluster-" + environment,
            endpoint_access=aws_eks.EndpointAccess.PUBLIC_AND_PRIVATE,
            default_capacity=0,
            vpc=vpc,
            vpc_subnets=[aws_ec2.SubnetSelection(subnets=[subnet_a_eks, subnet_b_eks, subnet_c_eks])],
            # issue with 3 subnet
            place_cluster_handler_in_vpc=True,
            version=cluster_version,
            cluster_handler_environment={
                "http_proxy": "http://login:pass@proxy.cloud.local:8080/"
            },
            kubectl_environment={
                "http_proxy": "http://login:pass@proxy.cloud.local:8080/"
            },
            security_group=eks_control_plane_sg,
            role=eks_control_plane_role,
        )

What did you expect to happen?

successful cluster creation

What actually happened?

Cloudwatch log of ProviderframeworkonEvent. function:
2021-01-09T14:37:27.905Z e785fa78-c5f8-471c-a495-59d75389a6c6 INFO [provider-framework] submit response to cloudformation { "Status": "FAILED", "Reason": "Error: connect ETIMEDOUT 63.32.73.253:443\n at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1107:14)", "StackId": "arn:aws:cloudformation:eu-west-1:accountid:stack/eks-stack-develop-cdk/276db000-5285-11eb-ab35-0615947f7f49", "RequestId": "ce8a03d7-fdf3-4def-acb4-6219fb352732", "PhysicalResourceId": "AWSCDK::CustomResourceProviderFramework::CREATE_FAILED", "LogicalResourceId": "clusterC5B25D0D" }

Proxy is fine i tested it against same endpoint ( strange that it calls ec2 and nor eks api

[ec2-user@ip-10-60-233-255 ~]$ curl -vk https://ec2-63-32-73-253.eu-west-1.compute.amazonaws.com.
* Rebuilt URL to: https://ec2-63-32-73-253.eu-west-1.compute.amazonaws.com./
* Uses proxy env variable https_proxy == 'http://user:password@proxy.cloud.local:8080/'
*   Trying 10.60.249.170...
* TCP_NODELAY set
* Connected to proxy.cloud.local (10.60.249.170) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to ec2-63-32-73-253.eu-west-1.compute.amazonaws.com:443
* Proxy auth using Basic with user 'user'
> CONNECT ec2-63-32-73-253.eu-west-1.compute.amazonaws.com:443 HTTP/1.1
> Host: ec2-63-32-73-253.eu-west-1.compute.amazonaws.com:443
> Proxy-Authorization: Basic token
> User-Agent: curl/7.61.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=lambda.eu-west-1.amazonaws.com
*  start date: Dec 23 00:00:00 2020 GMT
*  expire date: Jan 21 23:59:59 2022 GMT
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x17f8190)
> GET / HTTP/2
> Host: ec2-63-32-73-253.eu-west-1.compute.amazonaws.com
> User-Agent: curl/7.61.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 403 
< date: Tue, 12 Jan 2021 13:31:59 GMT
< content-length: 127
< x-amzn-requestid: c6c6badc-56de-4e6a-8266-0d7971505c84
< 
<MissingAuthenticationTokenException>
  <Message>Missing Authentication Token</Message>
</MissingAuthenticationTokenException>
* Connection #0 to host proxy.cloud.local left intact

Environment

  • **CDK CLI Version :1.83
  • Framework Version:
  • **Node.js Version: v14.13.0
  • **OS :macOS
  • **Language (Version): Python (3.9)

Other

I noticed that 5 lambda functions are created but only 1 of them "OnEventHandler" receives proxy configuration, but looks like it only one which interacts with api


This is 🐛 Bug Report

Metadata

Metadata

Assignees

Labels

@aws-cdk/aws-eksRelated to Amazon Elastic Kubernetes ServicebugThis issue is a bug.p1

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions