Unify Lambda Runtime using Runtime API#5306
Conversation
0b36615 to
bda1d72
Compare
305edca to
cc3ca2e
Compare
51b9b74 to
22f2fca
Compare
08634c7 to
955aced
Compare
955aced to
bd36f00
Compare
1b2bfa3 to
da655aa
Compare
680c0e9 to
bafbcc0
Compare
thrau
left a comment
There was a problem hiding this comment.
This looks fantastic guys! Really well done 🙌 Clean abstractions and mostly easy to understand. Can't wait to give this a try.
I just have a few really minor nits for this PR. :-)
|
|
||
| class InvokeSendError(Exception): | ||
| def __init__(self, invocation_id: str, payload: Optional[bytes]): | ||
| message = f"Error while trying to send invocation to RAPID for id {invocation_id}. Response: {payload}" |
There was a problem hiding this comment.
is it possible that we're printing control characters to stderr like this?
There was a problem hiding this comment.
Payload as well as invocation ids should be strings, so I see a slim chance of that happening here.
| executor_endpoint = Flask(f"executor_endpoint_{self.port}") | ||
|
|
||
| @executor_endpoint.route("/invocations/<req_id>/response", methods=["POST"]) | ||
| def invocation_response(req_id: str) -> ResponseReturnValue: | ||
| result = InvocationResult(req_id, request.data) | ||
| self.service_endpoint.invocation_result(invoke_id=req_id, invocation_result=result) | ||
| return Response(status=HTTPStatus.ACCEPTED) |
There was a problem hiding this comment.
this is a very creative solution! i can see how flask enabled the encapsulation of the HTTP server here. i think we could probably improve the code here by using our own Router implementation together with a werkzeug server. The inline methods are clearly necessary with flask, because the method decorators are bound to the flask app being dynamically created, but we could pull them out into methods of the ExecutorEndpoint with the Router class (or even using flask blueprints)
I would generally like to separate the HTTP endpoint implementation from the Server instance that spawns the HTTP server, for better composability and separation of concerns.
There was a problem hiding this comment.
Will be done in a follow-up PR! 👍
| def do_run(self) -> None: | ||
| endpoint = self._create_endpoint() | ||
| LOG.debug("Running executor endpoint API on %s:%s", self.host, self.port) | ||
| endpoint.run(self.host, self.port) |
There was a problem hiding this comment.
Since we're starting a new flask app for every lambda, i think this is also a good argument against flask. Using something more low-level like Router directly with werkzeug will likely reduce resource usage.
This is definitely out of scope for this revision though :-)
whummer
left a comment
There was a problem hiding this comment.
Fantastic set of changes, also can't wait to give this a try and see single-digit millisecond Lambda invocation times!! 🚀 😄
Agree with the comments around Flask usage for the individual executor endpoints, there we maybe have an opportunity for further improvement in future iterations.. 👍 Otherwise only two minor comments, but shouldn't hold back the merge. Great job!
| qualified_arn = function_version.id.qualified_arn() | ||
| version_manager = self.lambda_version_managers.get(qualified_arn) | ||
| if version_manager: | ||
| raise Exception("Version '%s' already created", qualified_arn) |
There was a problem hiding this comment.
| raise Exception("Version '%s' already created", qualified_arn) | |
| raise Exception(f"Version '{qualified_arn}' already created") |
There was a problem hiding this comment.
will pick these up in later stages 👍
|
|
||
| def start(self) -> None: | ||
| try: | ||
| invocation_thread = Thread(target=self.invocation_loop) |
There was a problem hiding this comment.
nit: we could set invocation_thread.daemon = True, to ensure the thread is terminated on process teardown (probably already covered by the logic around self.shutdown_event, but generally a good pattern to be on the safe side)
In this PR, we will create a re-implementation of the Lambda executors, to properly support multiple architectures as well as new runtimes, with the use of the Lambda Runtime API (https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html).
This is a draft PR for the progress.
To activate the new provider add
PROVIDER_OVERRIDE_LAMBDA=asf(note that it is currently not feature complete).This PR will automatically download the specified release of https://github.com/localstack/lambda-runtime-init as "Runtime Init".
Configuration options:
LAMBDA_PREBUILD_IMAGES=1(defaults to0) toggles the building of a lambda version specific container image that includes the code and init before any invoke happens instead of copying them into the container before each invoke. This is actually fairly simillar to the AWS behavior since they also have some initial "optimization" of Lambdas when creating a function. Unfortunately it involves a significant overhead in initial total execution time (from create to an invoke result), so for many single-use lambdas this heavily reduces performance.sequenceDiagram participant LS as LocalStack Endpoint participant RAPID as Runtime Init participant RIC as Runtime Interface Client LS-)RAPID: Start Container RAPID-)RIC: Start RIC alt Init success RIC->>RAPID: next() RAPID->>LS: Done else RIC->>RAPID: InitError RAPID->>LS: DoneError end loop per invocation LS-)RAPID: Invoke(id) RAPID-->>+RIC: next: Invocation(id) alt Invocation Success RIC->>RAPID: InvocationResult(id) else RIC->>RAPID: InvocationError(id) end par Rapid to LocalStack alt Invocation Success RAPID->>LS: Response(id) else RAPID->>LS: ErrorResponse(id) end and RIC to Rapid RIC->>-RAPID: next() end endflowchart TD API[Lambda API] SERVICE[Lambda Service] REPO[Repository] VM[Version Manager] ENV[Environment] EXEC[Executor] EXEC-CONN[Executor Connection] subgraph CONTAINER RAPID end API --> SERVICE SERVICE --> REPO SERVICE --> VM VM --> ENV ENV --> EXEC EXEC <--> EXEC-CONN EXEC-CONN -- invokes --> RAPID RAPID -- result/status --> EXEC-CONN EXEC -- starts/stops --> CONTAINER