Skip to content

Add foundation for basic xray integration#7892

Merged
dominikschubert merged 5 commits intomasterfrom
lambda-tracing-mode
Mar 21, 2023
Merged

Add foundation for basic xray integration#7892
dominikschubert merged 5 commits intomasterfrom
lambda-tracing-mode

Conversation

@dominikschubert
Copy link
Member

@dominikschubert dominikschubert commented Mar 17, 2023 •
edited by joe4dev
Loading

@dominikschubert dominikschubert requested a review from joe4dev March 17, 2023 09:47
@dominikschubert dominikschubert self-assigned this Mar 17, 2023
@dominikschubert dominikschubert temporarily deployed to localstack-ext-tests March 17, 2023 09:47 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Mar 17, 2023 •
edited
Loading

LocalStack integration with Pro

       3 files  ±  0         3 suites  ±0   1h 38m 39s ⏱️ - 3m 3s
1 832 tests +25  1 442 ✔️ +17  390 💤 +  8  0 ❌ ±0 
2 560 runs  +35  1 809 ✔️ +18  751 💤 +17  0 ❌ ±0 

Results for commit 30fcf75. ± Comparison against base commit 92dca7d.

♻️ This comment has been updated with latest results.

@dominikschubert dominikschubert marked this pull request as ready for review March 17, 2023 14:42
@dominikschubert dominikschubert requested a review from dfangl as a code owner March 17, 2023 14:42
@dominikschubert dominikschubert temporarily deployed to localstack-ext-tests March 17, 2023 14:42 — with GitHub Actions Inactive
@coveralls
Copy link

coveralls commented Mar 17, 2023 •
edited
Loading

Coverage Status

Coverage: 85.109% (-0.01%) from 85.122% when pulling 30fcf75 on lambda-tracing-mode into 92dca7d on master.

joe4dev
joe4dev requested changes Mar 20, 2023
View reviewed changes
Copy link
Member

@joe4dev joe4dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great foundation 🚀

I can confirm the test case tests.integration.awslambda.test_lambda_integration_xray.test_traceid_outside_handler works well on macOS with the custom RIE (x86).

Main changes before merging:

  1. Update Lambda RIE as this PR depends on localstack/lambda-runtime-init#14

@joe4dev
Copy link
Member

joe4dev commented Mar 20, 2023 •
edited
Loading

The AWS X-Ray examples (e.g., blank-python currently fail to deploy against LS due an internal server error:

2023-03-20T19:01:17.894  INFO --- [   asgi_gw_1] localstack.request.aws     : AWS cloudformation.CreateChangeSet => 500 (InternalError); CreateChangeSetInput({'StackName': 'blank-python', 'Capabilities': ['CAPABILITY_NAMED_IAM'], 'ChangeSetName': 'awscli-cloudformation-package-deploy-1679335277', 'Description': 'Created by AWS CLI at 2023-03-20T18:01:17.379796 UTC', 'ChangeSetType': 'CREATE'}, headers={'Host': 'localhost:4566', 'Accept-Encoding': 'identity', 'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'aws-cli/1.27.94 Python/3.10.8 Darwin/22.2.0 botocore/1.29.94', 'X-Amz-Date': '20230320T180117Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=test/20230320/us-east-1/cloudformation/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=391e285941a243edb33fdc7815f7ef4fcc2a622a11b23fb2afdb77cf827d7321', 'amz-sdk-invocation-id': 'c16e882f-3051-480d-afaf-3af1ab560cf8', 'amz-sdk-request': 'attempt=1', 'Content-Length': '1335', 'x-localstack-tgt-api': 'cloudformation', 'x-moto-account-id': '000000000000'}); InternalError(exception while calling cloudformation.CreateChangeSet: Traceback (most recent call last):
  File "/Users/joe/Projects/LocalStack/localstack/localstack/services/cloudformation/engine/template_preparer.py", line 142, in apply_serverless_transformation
    transformed = transform_sam(parsed_template, {}, loader)
  File "/Users/joe/Projects/LocalStack/localstack-ext/.venv/lib/python3.10/site-packages/samtranslator/translator/transform.py", line 37, in transform
    transformed = translator.translate(
  File "/Users/joe/Projects/LocalStack/localstack-ext/.venv/lib/python3.10/site-packages/samtranslator/translator/translator.py", line 135, in translate
    sam_plugins = prepare_plugins(self.plugins, parameter_values)
  File "/Users/joe/Projects/LocalStack/localstack-ext/.venv/lib/python3.10/site-packages/samtranslator/translator/translator.py", line 425, in prepare_plugins
    required_plugins.insert(0, ServerlessAppPlugin(parameters=parameters))
  File "/Users/joe/Projects/LocalStack/localstack/localstack/utils/patch.py", line 38, in proxy
    return new(target, *args, **kwargs)
  File "/Users/joe/Projects/LocalStack/localstack-ext/localstack_ext/services/cloudformation/cloudformation_extended.py", line 113, in serverless_app_init
    self._sar_client = aws_stack.connect_to_service("serverlessrepo")
AttributeError: can't set attribute '_sar_client'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/chain.py", line 90, in handle
    handler(self, self.context, response)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/handlers/service.py", line 122, in __call__
    handler(chain, context, response)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/handlers/service.py", line 92, in __call__
    skeleton_response = self.skeleton.invoke(context)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/skeleton.py", line 153, in invoke
    return self.dispatch_request(context, instance)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/skeleton.py", line 165, in dispatch_request
    result = handler(context, instance) or {}
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/skeleton.py", line 117, in __call__
    return self.fn(*args, **kwargs)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/aws/api/core.py", line 163, in operation_marker
    return fn(*args, **kwargs)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/services/cloudformation/provider.py", line 465, in create_change_set
    template = template_preparer.transform_template(template, parameters, stack=stack)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/services/cloudformation/engine/template_preparer.py", line 64, in transform_template
    result = apply_serverless_transformation(result)
  File "/Users/joe/Projects/LocalStack/localstack/localstack/services/cloudformation/engine/template_preparer.py", line 145, in apply_serverless_transformation
    raise FailedTransformationException(transformation=SERVERLESS_TRANSFORM, message=str(e))
localstack.services.cloudformation.engine.template_preparer.FailedTransformationException: can't set attribute '_sar_client'

Maybe related to the AWS client rework and/or IAM 🤔

Update: Works against master and rebasing fixed it after:

  • Configuring endpoint_url: client = boto3.client('lambda', endpoint_url=f"http://{os.environ['LOCALSTACK_HOSTNAME']}:{os.environ['EDGE_PORT']}")
  • disable init port mapping container_config.ports.add(config.LAMBDA_INIT_DELVE_PORT, config.LAMBDA_INIT_DELVE_PORT)

@joe4dev
Copy link
Member

joe4dev commented Mar 20, 2023

We should test the retry behavior with a failing function. Just ran into a case where unlimited retries created traces many "TracesProcessedCount": 121,

@joe4dev
Copy link
Member

joe4dev commented Mar 20, 2023

Observation: Telemetry records are continuously sent via AWS xray.PutTelemetryRecords => 200 (that might get verbose with provisioned concurrency)
telemetry

@dominikschubert dominikschubert temporarily deployed to localstack-ext-tests March 21, 2023 10:03 — with GitHub Actions Inactive
@joe4dev
Copy link
Member

joe4dev commented Mar 21, 2023

✔️ The X-Ray test tests.integration.awslambda.test_lambda_xray.TestLambdaXrayIntegration.test_basic_xray_integration succeeds with the latest RIE v0.1.13-pre on macOS for both x86_64 and arm64 (after commenting mypy_boto3_*)

@dominikschubert dominikschubert temporarily deployed to localstack-ext-tests March 21, 2023 13:31 — with GitHub Actions Inactive
@dominikschubert dominikschubert temporarily deployed to localstack-ext-tests March 21, 2023 13:41 — with GitHub Actions Inactive
@dominikschubert dominikschubert temporarily deployed to localstack-ext-tests March 21, 2023 13:41 — with GitHub Actions Inactive
@dominikschubert dominikschubert requested a review from joe4dev March 21, 2023 13:43
joe4dev
joe4dev approved these changes Mar 21, 2023
View reviewed changes
Copy link
Member

@joe4dev joe4dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎉 Looking forward to basic X-Ray support 📈

I also ran some tests without pro activated (+exception throwing) and it looks fine as well.

@dominikschubert dominikschubert merged commit 814bbe8 into master Mar 21, 2023
@dominikschubert dominikschubert deleted the lambda-tracing-mode branch March 21, 2023 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joe4dev joe4dev joe4dev approved these changes

@dfangl dfangl Awaiting requested review from dfangl

Assignees

@dominikschubert dominikschubert

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dominikschubert @coveralls @joe4dev