Task description
There are tests across both unit and functional tests that intermittently fail, particularly when processing speeds are slow and timeout thresholds are hit (although this is not the only flaky test failure mode).
Examples of flaky failures
1. Test cleanup failure - AEPEdgeFunctionalTests.swift
testSendEvent_edgeNetworkResponseContainsLocationHint_sendEventDoesNotIncludeExpiredLocationHint
runs before
testSendEvent_fatalError
The latter case fails with an unexpected locationHint:result event in the setup process - a locationHint:result type event is the mocked response from the former case
testSendEvent_twoConsecutiveCalls_resetBefore_appendsReceivedClientSideStore runs before
testSendEvent_twoConsecutiveCalls_resetInBetween_doesNotAppendReceivedClientSideStore
The latter case fails with an unexpected state:store event in the setup process - a state:store type event is the mocked response from the former case
See job failure
2. Test setup timeout failure - AEPEdgeFunctionalTests.swift
|
func testSendEvent_twoConsecutiveCalls_appendsReceivedClientSideStore() { |
Fails with: XCTAssertFalse failed - Timed out waiting for event type com.adobe.eventType.hub and source com.adobe.eventSource.sharedState, expected 4, but received 1
Hypothesis is the setup process doesn't have enough time to process all the shared state events before the timeout triggers, especially on slower machines
See job failure
testSendEvent_fatalError
XCTAssertFalse failed - Timed out waiting for event type com.adobe.eventType.hub and source com.adobe.eventSource.sharedState, expected 4, but received 1
See job failure
testSendEvent_fatalError400
XCTAssertFalse failed - Timed out waiting for event type com.adobe.eventType.configuration and source com.adobe.eventSource.responseContent, expected 1, but received 0
See job failure
3. Event listener timeout failure - UnitTests EdgePublicAPITests
testGetLocationHint_dispatchesEdgeRequestIdentity_receivesResponseIdentity_withEmptyHint
|
func testGetLocationHint_dispatchesEdgeRequestIdentity_receivesResponseIdentity_withEmptyHint() { |
Fails on
XCTAssertNil failed: "AEPError"
See job failure
testGetLocationHint_dispatchesEdgeRequestIdentity_receivesResponseIdentity_withValidHint
|
func testGetLocationHint_dispatchesEdgeRequestIdentity_receivesResponseIdentity_withValidHint() { |
Fails on
|
XCTAssertEqual("or2", hint) |
4. MockDataQueue empty failure - UnitTests EdgeExtensionsTests
|
func testHandleExperienceEventRequest_validData_consentYes_queues() { |
Fails on
|
XCTAssertEqual(1, mockDataQueue.count()) |
XCTAssertEqual failed: ("1") is not equal to ("0")
See job failure
Additional implementation details or code snippet(s)
No response
Task description
There are tests across both unit and functional tests that intermittently fail, particularly when processing speeds are slow and timeout thresholds are hit (although this is not the only flaky test failure mode).
Examples of flaky failures
1. Test cleanup failure -
AEPEdgeFunctionalTests.swifttestSendEvent_edgeNetworkResponseContainsLocationHint_sendEventDoesNotIncludeExpiredLocationHintruns before
testSendEvent_fatalErrorThe latter case fails with an unexpected
locationHint:resultevent in the setup process - alocationHint:resulttype event is the mocked response from the former casetestSendEvent_twoConsecutiveCalls_resetBefore_appendsReceivedClientSideStoreruns beforetestSendEvent_twoConsecutiveCalls_resetInBetween_doesNotAppendReceivedClientSideStoreThe latter case fails with an unexpected
state:storeevent in the setup process - astate:storetype event is the mocked response from the former caseSee job failure
2. Test setup timeout failure -
AEPEdgeFunctionalTests.swiftaepsdk-edge-ios/Tests/FunctionalTests/AEPEdgeFunctionalTests.swift
Line 486 in 2a7c7b2
Fails with: XCTAssertFalse failed - Timed out waiting for event type com.adobe.eventType.hub and source com.adobe.eventSource.sharedState, expected 4, but received 1
Hypothesis is the setup process doesn't have enough time to process all the shared state events before the timeout triggers, especially on slower machines
See job failure
testSendEvent_fatalErrorXCTAssertFalse failed - Timed out waiting for event type com.adobe.eventType.hub and source com.adobe.eventSource.sharedState, expected 4, but received 1
See job failure
testSendEvent_fatalError400XCTAssertFalse failed - Timed out waiting for event type com.adobe.eventType.configuration and source com.adobe.eventSource.responseContent, expected 1, but received 0
See job failure
3. Event listener timeout failure - UnitTests
EdgePublicAPITeststestGetLocationHint_dispatchesEdgeRequestIdentity_receivesResponseIdentity_withEmptyHint
aepsdk-edge-ios/Tests/UnitTests/EdgePublicAPITests.swift
Line 153 in 2a7c7b2
Fails on
aepsdk-edge-ios/Tests/UnitTests/EdgePublicAPITests.swift
Line 166 in 2a7c7b2
XCTAssertNil failed: "AEPError"
See job failure
testGetLocationHint_dispatchesEdgeRequestIdentity_receivesResponseIdentity_withValidHint
aepsdk-edge-ios/Tests/UnitTests/EdgePublicAPITests.swift
Line 131 in 2a7c7b2
Fails on
aepsdk-edge-ios/Tests/UnitTests/EdgePublicAPITests.swift
Line 143 in 2a7c7b2
4. MockDataQueue empty failure - UnitTests
EdgeExtensionsTestsaepsdk-edge-ios/Tests/UnitTests/EdgeExtensionTests.swift
Line 174 in 2a7c7b2
Fails on
aepsdk-edge-ios/Tests/UnitTests/EdgeExtensionTests.swift
Line 180 in 2a7c7b2
XCTAssertEqual failed: ("1") is not equal to ("0")
See job failure
Additional implementation details or code snippet(s)
No response