Skip to content

When multiple Lambda functions share code, build a single asset shared between them #989

@tomfaulhaber

Description

@tomfaulhaber

Use case:

In order to build lambdas that use the Python science stack, we need to build a giant (50+MB) zip file with all the dependencies. We then add all of our Python functions to that zip and specify the same zip as the code for each lambda with a different handler.

The problem:

Each of the lambda functions that I create have a unique asset object (even if I create a single, shared lambda.Code object). Each of these objects creates an S3 object. So if I have ten lambda functions, I will upload ten separate asset objects at 50+ MB each.

Potential solutions:

  1. Make each lambda.AssetCode object unique by checking to see if asset is already set before creating a new one in bind(). This means that users would need to explicitly create the shared Code object and reuse it in each lambda definition.
  2. Make it unique to the path in the lambda.AssetCode by adding a map to lambda.AssetCode and pulling the same asset into multiple AssetCode objects. In this scenario, users could simply reference the same file/directory.
  3. Build the solution lower down in the assets.Asset class. In this example, we would cache the path and make sure that we only do the upload once.

I think the last solution is the only real solution, because the other two solutions don't handle the case where you create two lambdas, but only wire the second one into your app which is likely to happen in various scenarios.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions