Skip to content

opensearch,elasticsearch,events-targets: Custom Resources provided in these modules always attempt to install latest version of AWS SDK v2 #23113

@mrgrain

Description

@mrgrain

Describe the bug

The following built-in Custom Resources force the install of the latest AWS SDK version in their Lambda which will fail in environments with restricted internet access (e.g. China Regions or when internet access is disabled in Lambdas):

  • OpenSearchAccessPolicy in aws-elasticsearch
  • ElasticsearchAccessPolicy in aws-opensearch
  • LogGroupResourcePolicy in aws-elasticsearch, aws-opensearch and aws-events-targets

There is no way to disable this behavior, because the resources extend AwsCustomResource which defaults installLatestAwsSdk to true. They do not provide an option to disable this.

Expected Behavior

The affected resources do not attempt to install the latest SDK version. All API request for these resources are known so it's not required to have the latest version available.

Current Behavior

They always attempt to install the latest SDK version.

Reproduction Steps

class TestStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const domainProps: opensearch.DomainProps = {
      removalPolicy: RemovalPolicy.DESTROY,
      version: opensearch.EngineVersion.ELASTICSEARCH_7_1,
      ebs: {
        volumeSize: 10,
        volumeType: EbsDeviceVolumeType.GENERAL_PURPOSE_SSD,
      },
      logging: {
        slowSearchLogEnabled: true,
        appLogEnabled: true,
      },
      nodeToNodeEncryption: true,
      encryptionAtRest: {
        enabled: true,
      },
      advancedOptions: {
        'rest.action.multi.allow_explicit_index': 'false',
        'indices.fielddata.cache.size': '25',
        'indices.query.bool.max_clause_count': '2048',
      },
      // test the access policies custom resource works
      accessPolicies: [
        new iam.PolicyStatement({
          effect: iam.Effect.ALLOW,
          actions: ['es:ESHttp*'],
          principals: [new iam.AccountRootPrincipal()],
          resources: ['*'],
        }),
      ],
    };

    // create 2 domains to ensure that Cloudwatch Log Group policy names dont conflict
    new opensearch.Domain(this, 'Domain1', domainProps);
    new opensearch.Domain(this, 'Domain2', domainProps);
  }
}

Possible Solution

  • Check if we can default the setting to false for these custom resources
    It should be possible to ascertain if the used APIs are available in the default SDK
    Then add installLatestAwsSdk: false to here
  • Use a layer to provide the latest version of the SDK
    Similar to how we provide the AWS CLI already

Additional Information/Context

Workaround:

declare stack: cdk.Stack;

// Create an Aspect to stop installing the latest SDK version
class AwsCustomResourceUseDefaultAwsSdk implements cdk.IAspect {
  public readonly resourceTypes: string[];

  public constructor(resourceTypes: string[] = ['Custom::AWS']) {
    this.resourceTypes = resourceTypes;
  }

  public visit(node: IConstruct): void {
    if (node instanceof cdk.CfnResource && this.resourceTypes.includes(node.cfnResourceType)) {
      node.addPropertyOverride('InstallLatestAwsSdk', false);
    }
  }
}

// Apply this Aspect to any Stack
// Note that specific resource types have to be provided in the constructor call
cdk.Aspects.of(stack).add(new AwsCustomResourceUseDefaultAwsSdk(['Custom::OpenSearchAccessPolicy']));

CDK CLI Version

2.52.20

Framework Version

2.52.0

Node.js Version

any

OS

macos

Language

Typescript, Python, .NET, Java, Go

Language Version

No response

Other information

These Custom Resources currently run in nodejs14.x which defaults the AWS SDK for JS to version 2.1055.0 (source).

I have checked and confirm that the API for OpenSearchAccessPolicy & ElasticSearchAccessPolicy is available in this particular version of the SDK.

Metadata

Metadata

Labels

@aws-cdk/custom-resourcesRelated to AWS CDK Custom ResourcesbugThis issue is a bug.effort/mediumMedium work item – several days of effortp1

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions