Skip to content

Azure Functions Instrumentation#1613

Merged
kevingosse merged 8 commits into
masterfrom
az-function-integration
Sep 22, 2021
Merged

Azure Functions Instrumentation#1613
kevingosse merged 8 commits into
masterfrom
az-function-integration

Conversation

@colin-higgins

Copy link
Copy Markdown
Member

Introduces the Azure Function integration

@DataDog/apm-dotnet

@colin-higgins colin-higgins added the status:work-in-progress Actively worked on. If this is a PR, no review needed yet. WIP. label Jul 19, 2021
@colin-higgins colin-higgins self-assigned this Jul 19, 2021
@colin-higgins colin-higgins changed the title Az function integration Azure Function Instrumentation Jul 20, 2021

@lucaspimentel lucaspimentel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Half-review. Will continue tomorrow 😄

Comment thread .gitignore Outdated
@jaycdave88 jaycdave88 requested a review from kevingosse July 28, 2021 14:48
@lucaspimentel lucaspimentel requested a review from a team July 28, 2021 14:53

@lucaspimentel lucaspimentel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continued from last night. LGTM 👍🏽, my comments are mostly nitpicking.

Comment thread src/Datadog.Trace/Tags.cs Outdated
Comment thread src/Datadog.Trace/Configuration/IntegrationIds.cs Outdated
Comment thread src/Datadog.Trace/Tagging/AzureFunctionTags.cs Outdated

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me so far 🙂

Comment thread integrations.json Outdated
Comment thread src/Datadog.Trace/DiagnosticListeners/AspNetCoreDiagnosticObserver.cs Outdated
Comment thread src/Datadog.Trace/SpanTypes.cs Outdated
Comment thread src/Datadog.Trace/Tagging/AzureFunctionTags.cs Outdated
@lucaspimentel lucaspimentel requested a review from a team July 28, 2021 15:53

@zacharycmontoya zacharycmontoya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far so good 👍🏼

@colin-higgins colin-higgins force-pushed the az-function-integration branch from 5b3ad5c to a46e5bb Compare August 4, 2021 14:21
Comment thread test/test-applications/azure-functions/Directory.Build.props Outdated
Comment thread test/test-applications/azure-functions/Directory.Build.props Outdated
@colin-higgins colin-higgins force-pushed the az-function-integration branch from b575231 to 5ad7092 Compare August 11, 2021 19:41
@colin-higgins colin-higgins marked this pull request as ready for review August 11, 2021 19:50

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, mostly minor comments 🙂

Comment thread src/Datadog.Trace/PlatformHelpers/AzureAppServices.cs Outdated

internal Scope Parent { get; }

internal Scope Root => Parent?.Root ?? this;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So satisfying 😉

Comment on lines +25 to +26
public static async Task TriggerAllTimer([TimerTrigger(EveryTenSeconds)] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
await CallFunctionHttp("trigger");
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we able to run integration tests for the timer/http triggers, as they don't require any Azure infra?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I plan to include those in a follow up, but I will try to get them in today.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not successfully run the functions host in our integration test framework yet, but the integration is running on several sample apps and I will monitor them very closely and continue work on the automation.

Comment thread Datadog.Trace.sln Outdated
Comment thread src/Datadog.Trace/Tagging/AzureFunctionsTags.cs
Comment thread src/Datadog.Trace/DiagnosticListeners/AspNetCoreDiagnosticObserver.cs Outdated

@zacharycmontoya zacharycmontoya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! 👍🏼

@lucaspimentel lucaspimentel changed the title Azure Function Instrumentation Azure Functions Instrumentation Aug 26, 2021
@colin-higgins colin-higgins force-pushed the az-function-integration branch 2 times, most recently from c66d69a to e33216a Compare September 2, 2021 14:43
@colin-higgins colin-higgins removed the status:work-in-progress Actively worked on. If this is a PR, no review needed yet. WIP. label Sep 2, 2021
Comment thread src/Datadog.Trace.ClrProfiler.Native/environment_variables.h Outdated
Comment thread integrations.json Outdated
Comment thread test/Datadog.Trace.TestHelpers/ProfilerHelper.cs Outdated
Comment thread src/Datadog.Trace.ClrProfiler.Native/cor_profiler.cpp Outdated
scope = tracer.StartActive(OperationName);
foreach (var tagProperty in AzureFunctionsTags.AzureFunctionsExtraTags)
{
scope.Root.Span.SetTag(tagProperty.Key, tagProperty.Getter(tags));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this else happen often, or are these spans usually roots? If this is common, we should consider a custom tags implementation (like AzureFunctionsTags above) to avoid using Span.SetTag() when possible.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This happens for all HttpTriggers, basically anything that enters the diagnostic observer

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, thanks. It's not necessarily a blocker for this PR, but something to consider (later?) to improve performance.

@lucaspimentel lucaspimentel added the area:automatic-instrumentation Automatic instrumentation managed C# code (Datadog.Trace.ClrProfiler.Managed) label Sep 7, 2021
@colin-higgins colin-higgins force-pushed the az-function-integration branch from bcd59bb to 3bb6afe Compare September 7, 2021 19:32
@colin-higgins colin-higgins force-pushed the az-function-integration branch from 0514eb6 to 413c506 Compare September 13, 2021 13:25
private static readonly IDatadogLogger Log = DatadogLogging.GetLoggerFor(typeof(AzureFunctionsCommon));

public static CallTargetState OnMethodBegin<TTarget>(TTarget instance, IFunctionInstance instanceParam)
{

@tonyredondo tonyredondo Sep 15, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use the generic constraint for IFunctionInstance, that will remove the boxing of the proxy struct. The same for other methods.

Comment thread tracer/test/test-applications/azure-functions/Directory.Build.props
Comment thread tracer/src/Datadog.Trace/Tags.cs Outdated
tags = null;

if (!tracer.Settings.IsIntegrationEnabled(integrationId) || HttpBypassHelper.UriContainsAnyOf(requestUri, tracer.Settings.HttpClientExcludedUrlSubstrings))
if (!tracer.Settings.IsIntegrationEnabled(integrationId) || PlatformHelpers.PlatformStrategy.ShouldSkipClientSpan(tracer.ActiveScope) || HttpBypassHelper.UriContainsAnyOf(requestUri, tracer.Settings.HttpClientExcludedUrlSubstrings))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows strategic exclusion of http spans.

SiteKind = "functionapp";
SiteType = "function";
IsFunctionsApp = true;
PlatformStrategy.ShouldSkipClientSpan = ShouldSkipClientSpanWithinFunctions;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where the azure function http skip strategy is added

{
internal static class PlatformStrategy
{
private static Func<Scope, bool> _shouldSkipClientSpan = (s) => false;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default strategy, able to be overridden.

?? 20_000;
#else
?? 100;
?? 500;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed in azure functions that higher load scenarios meant the initial requests could take much longer than 100ms and this resulted in a failure and a retry loop on startup. Increasing to 500ms I noticed no timeouts.

@colin-higgins colin-higgins force-pushed the az-function-integration branch from b7b90a1 to bf40f30 Compare September 21, 2021 17:31
Log.Information("IsProfilerAttached: true");

var asm = typeof(Instrumentation).Assembly;
Log.Information($"[Assembly metadata] Location: {asm.Location}");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be debug logs or did you want to keep them as info logs?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want these to show up everytime we load Datadog.Trace
They seem invaluable to me for troubleshooting.

@zacharycmontoya zacharycmontoya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM again

@andrewlock

Copy link
Copy Markdown
Member

Code Coverage Report 📊

✔️ Merging #1613 into master will not change line coverage
✔️ Merging #1613 into master will not change branch coverage
⛔ Merging #1613 into master will will increase complexity by 77

master #1613 Change
Lines 9329 / 14796 9431 / 14999
Lines % 63% 63% 0% ✔️
Branches 4137 / 6452 4165 / 6512
Branches % 64% 64% 0% ✔️
Complexity 7596 7673 77

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #1613 Change
Lines % 63% 63% 0% ✔️
Branches % 64% 64% 0% ✔️
Complexity 7596 7673 77

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Scope -6% -67% 4

The following classes were added in #1613:

File Line coverage Branch coverage Complexity
Datadog.Trace.ClrProfiler.AutoInstrumentation.AspNet.ThreadContext_AssociateWithCurrentThread_Integration 100% 100% 4
Datadog.Trace.ClrProfiler.AutoInstrumentation.AspNet.ThreadContext_DisassociateFromCurrentThread_Integration 100% 100% 2
Datadog.Trace.ClrProfiler.AutoInstrumentation.Azure.Functions.AzureFunctionsCommon 0% 0% 25
Datadog.Trace.ClrProfiler.AutoInstrumentation.Azure.Functions.AzureFunctionsExecutorTryExecuteAsyncIntegration 0% 0% 3
Datadog.Trace.ClrProfiler.AutoInstrumentation.Azure.Functions.FunctionInvocationMiddlewareInvokeIntegration 0% 0% 3
...And 4 more

View the full reports for further details:

@kevingosse kevingosse merged commit 686f6a0 into master Sep 22, 2021
@kevingosse kevingosse deleted the az-function-integration branch September 22, 2021 15:02
@github-actions github-actions Bot added this to the vNext milestone Sep 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:automatic-instrumentation Automatic instrumentation managed C# code (Datadog.Trace.ClrProfiler.Managed) type:new-feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants