Skip to content

Flaky Tests #2279

@TimothyMothra

Description

@TimothyMothra

The following tests are flaky and affecting our NetCore builds in Linux

tests

  • ResumesTransmissionSenderAfterPauseDurationWhenTransmissionGetServerError
  • ResumesTransmissionSenderAfterPauseDurationWhenTransmissionGetServerUnavailable
  • ResumesTransmissionSenderAfterPauseDurationWhenTransmissionTimesOut
  • ProactivelySampledInTelemetryCapturedWhenProactiveSamplingRateIsLowerThanTarget
  • SendingLogicMarkedAsInternalSdkOperation
  • ResumesTransmissionSenderAfterPauseDurationForBadGateway
  • WaitsUntilTelemetryBufferIsSafeToModify
  • FlushesBufferWhenNumberOfTelemetryItemsReachesMax

these tests from the Web project consistently fail on NetFramework

  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentReturnsCorrectDeploymentId
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleInstanceReturnAnInstanceIdThatMatchesAnAzureInstanceId
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentReturnsTheCurrentRoleInstanceWhichIsNotNull
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.ServiceRuntimeProducesARoleEnvironmentObject
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleInstanceReturnsARollWhichIsNotNull
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleReturnsCorrectName
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentReturnsCorrectAvailabilityState
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentIsAvailableReturnsFalseIfServiceRuntimeDoesntExit

test class

  • ServerTelemetryChannelE2ETests

other

#1671

[xUnit.net 00:00:05.4325976]       actual.Duration '00:00:00.0387438' is not less than timer.Elpased '00:00:00.0154093'
[xUnit.net 00:00:05.4326745]       Expected: True
[xUnit.net 00:00:05.4327267]       Actual:   False
##[error][xUnit.net 00:00:05.4309248]     WebApi.FunctionalTests.FunctionalTest.ExceptionTelemetryWebApiTests.TestBasicRequestPropertiesAfterRequestingControllerThatThrows [FAIL]
[xUnit.net 00:00:05.4337861]       Stack Trace:
[xUnit.net 00:00:05.4351258]         E:\A\_work\20\s\NETCORE\test\FunctionalTestUtils\TelemetryTestsBase.cs(64,0): at FunctionalTestUtils.TelemetryTestsBase.ValidateBasicRequest(InProcessServer server, String requestPath, RequestTelemetry expected)
[xUnit.net 00:00:05.4354944]         E:\A\_work\20\s\NETCORE\test\WebApi.FunctionalTests\FunctionalTest\ExceptionTelemetryWebApiTests.cs(33,0): at WebApi.FunctionalTests.FunctionalTest.ExceptionTelemetryWebApiTests.TestBasicRequestPropertiesAfterRequestingControllerThatThrows()
[xUnit.net 00:00:05.7104819]   Finished:    WebApi.FunctionalTests.Tests

#1037

2018-12-28T22:39:32.2777167Z Starting test execution, please wait...
2018-12-28T22:40:23.4984741Z Failed   Microsoft.ApplicationInsights.WindowsServer.Channel.AdaptiveSamplingTelemetryProcessorTest.SamplingPercentageAdjustsAccordingToConstantHighProductionRate
2018-12-28T22:40:23.4985603Z Error Message:
2018-12-28T22:40:23.4985771Z  Assert.IsTrue failed. 
2018-12-28T22:40:23.4987663Z Stack Trace:
2018-12-28T22:40:23.4988582Z    at Microsoft.ApplicationInsights.WindowsServer.Channel.AdaptiveSamplingTelemetryProcessorTest.SamplingPercentageAdjustsAccordingToConstantHighProductionRate() in /mnt/a/1/s/Test/ServerTelemetryChannel.Test/Shared.Tests/AdaptiveSamplingTelemetryProcessorTest.cs:line 128
2018-12-28T22:40:23.4989225Z 
2018-12-28T22:40:23.4990530Z Standard Output Messages:
2018-12-28T22:40:23.4990780Z  
2018-12-28T22:40:23.4990970Z  
2018-12-28T22:40:23.4991151Z  Debug Trace:
2018-12-28T22:40:23.4992222Z  [Sampling% evaluation] 2018-12-28T22:39:59.4728938+00:00, Eps: 20, Current %: 100, New %: 25, Changed: False
2018-12-28T22:40:23.4992867Z  [Sampling% evaluation] 2018-12-28T22:40:00.4723852+00:00, Eps: 20.5, Current %: 100, New %: 20, Changed: False
2018-12-28T22:40:23.4993395Z  [Sampling% evaluation] 2018-12-28T22:40:01.4727203+00:00, Eps: 20.375, Current %: 100, New %: 20, Changed: True
2018-12-28T22:40:23.4993927Z  [Sampling% evaluation] 2018-12-28T22:40:02.4739779+00:00, Eps: 4, Current %: 20, New %: 25, Changed: False
2018-12-28T22:40:23.4994406Z  [Sampling% evaluation] 2018-12-28T22:40:03.4702612+00:00, Eps: 3.75, Current %: 20, New %: 25, Changed: False
2018-12-28T22:40:23.4994919Z  [Sampling% evaluation] 2018-12-28T22:40:04.4701655+00:00, Eps: 4.3125, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4995400Z  [Sampling% evaluation] 2018-12-28T22:40:05.4708253+00:00, Eps: 4.734375, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4995927Z  [Sampling% evaluation] 2018-12-28T22:40:06.4749983+00:00, Eps: 4.55078125, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4996416Z  [Sampling% evaluation] 2018-12-28T22:40:07.4757375+00:00, Eps: 4.9130859375, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4996954Z  [Sampling% evaluation] 2018-12-28T22:40:08.4715758+00:00, Eps: 4.434814453125, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4997771Z  [Sampling% evaluation] 2018-12-28T22:40:09.4718122+00:00, Eps: 3.82611083984375, Current %: 20, New %: 25, Changed: True
2018-12-28T22:40:23.4998604Z  [Sampling% evaluation] 2018-12-28T22:40:10.4719975+00:00, Eps: 1, Current %: 25, New %: 100, Changed: False
2018-12-28T22:40:23.4999198Z  [Sampling% evaluation] 2018-12-28T22:40:11.4754562+00:00, Eps: 1.25, Current %: 25, New %: 100, Changed: True
2018-12-28T22:40:23.4999748Z  [Sampling% evaluation] 2018-12-28T22:40:12.4760677+00:00, Eps: 20, Current %: 100, New %: 25, Changed: False
2018-12-28T22:40:23.5000204Z  [Sampling% evaluation] 2018-12-28T22:40:13.4687556+00:00, Eps: 20, Current %: 100, New %: 25, Changed: False
2018-12-28T22:40:23.5000736Z  [Sampling% evaluation] 2018-12-28T22:40:14.4733533+00:00, Eps: 20, Current %: 100, New %: 25, Changed: True
2018-12-28T22:40:23.5001716Z  [Sampling% evaluation] 2018-12-28T22:40:15.4698518+00:00, Eps: 7, Current %: 25, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5002287Z  [Sampling% evaluation] 2018-12-28T22:40:16.4694474+00:00, Eps: 6, Current %: 25, New %: 20, Changed: False
2018-12-28T22:40:23.5002762Z  [Sampling% evaluation] 2018-12-28T22:40:17.4740224+00:00, Eps: 7, Current %: 25, New %: 16.6666666666667, Changed: True
2018-12-28T22:40:23.5003310Z  [Sampling% evaluation] 2018-12-28T22:40:18.4702060+00:00, Eps: 6, Current %: 16.6666666666667, New %: 12.5, Changed: False
2018-12-28T22:40:23.5003797Z  [Sampling% evaluation] 2018-12-28T22:40:19.4708565+00:00, Eps: 5, Current %: 16.6666666666667, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5004364Z  [Sampling% evaluation] 2018-12-28T22:40:20.4709985+00:00, Eps: 4.5, Current %: 16.6666666666667, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5004852Z  [Sampling% evaluation] 2018-12-28T22:40:21.4702893+00:00, Eps: 4.375, Current %: 16.6666666666667, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5005423Z  [Sampling% evaluation] 2018-12-28T22:40:22.4703054+00:00, Eps: 4.03125, Current %: 16.6666666666667, New %: 20, Changed: True
2018-12-28T22:40:23.5005872Z  [Sampling% evaluation] 2018-12-28T22:40:23.4702887+00:00, Eps: 4, Current %: 20, New %: 25, Changed: False
2018-12-28T22:40:23.5006317Z  'Ideal' telemetry item count: 125
2018-12-28T22:40:23.5006481Z  Expected range: from 63 to 187
2018-12-28T22:40:23.5006673Z  Actual telemetry item count: 201 (160.8% of ideal)
2018-12-28T22:40:23.5006781Z 

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions