Add array flattening support by timkimadobe · Pull Request #1152 · adobe/aepsdk-core-ios

timkimadobe · 2025-06-20T00:25:23Z

Description

This PR implements array flattening support, where array indexes are placed in the key path using dot notation. This array flattening is used whenever top level or nested array structures need to be flattened to a single-level dictionary of key value pairs. It would affect use cases like:

Rules conditions and consequences
Event hash generation

Given

{
  "array": [1, 2],
  "nested": {"a": "b"},
  "key": "value"
}

Original JSON Key	Before	After	Notes
`array`	`"array": [1, 2]`	`"array.0": 1` `"array.1": 2`	NEW: Array items are expanded with index keys using dot notation
`nested`	`"nested.a": "b"`	`"nested.a": "b"`	Unchanged
`key`	`"key": "value"`	`"key": "value"`	Unchanged

It also implements:

A backwards compatible carve out for the token ~all_url so that arrays remain unflattened and preserve the existing behavior.
- Achieved by implementing a flag in dictionary flattening which controls whether or not to flatten inner arrays.
- Updated test cases which validate the backwards compatible ~all_url handling.
Unit test cases for the dictionary and array flattening logic
Functional test case for rules engine correctly interpreting and applying rules condition and consequence using the new array notation

Impact

Explanation taken from (thank you @spoorthipujariadobe!):

Add support for nested lists to map flattening logic aepsdk-core-android#777

This change affects how event data map is flattened in:

Event data key lookup for rules matcher condition: Since key lookup in nested lists was never supported, this is a net new feature for this case and shouldn't create any backwards compatibility issues.
Event data hash generation for event history read and writes: Messaging SDK was the only extension to use event history and it never recorded maps with nested lists so this change should not have any impact.
Converting event data to encoded URL format using ~all_url: Unsure if customers use this in their Launch rules and what impact this change might have. Please share if this is a breaking change for this case.

Related Issue

Motivation and Context

Explanation taken from (thank you @spoorthipujariadobe!):

Add support for nested lists to map flattening logic aepsdk-core-android#777

With the existing logic, if a map contains a nested list, it is considered as unsupported for flattening and the resulting flattened map has the list added as an object, making it impossible to lookup for an element at a specific index in the list for rules matcher condition.

We need this ability to support content card disqualification when the card is dismissed. The rule condition in this case verifies that the dismiss event happened on the specific card we want to disqualify and writes a disqualify event to event history as a consequence. To verify this card association, we need to look up and match the activity id which is present inside the propositions array in the dismiss event data.

Disqualify rule example:


{
    "condition": {
        "type": "group",
        "definition": {
            "logic": "and",
            "conditions": [
                {
                    "type": "group",
                    "definition": {
                        "logic": "and",
                        "conditions": [
                             {
                                "definition": {
                                    "key": "~type",
                                    "matcher": "eq",
                                    "values": [
                                        "com.adobe.eventType.edge"
                                    ]
                                },
                                "type": "matcher"
                            },
                            {
                                "definition": {
                                    "key": "~source",
                                    "matcher": "eq",
                                    "values": [
                                        "com.adobe.eventSource.requestContent"
                                    ]
                                },
                                "type": "matcher"
                            },
                            {
                                "definition": {
                                    "key": "xdm.eventType",
                                    "matcher": "eq",
                                    "values": [
                                        "decisioning.propositionDismiss"
                                    ]
                                },
                                "type": "matcher"
                            },
                            {
                                "definition": {
                                    "key": "xdm._experience.decisioning.propositions.0.scopeDetails.activity.id",
                                    "matcher": "eq",
                                    "values": [
                                        "a43122c4-bf19-499f-b507-087a028d1769#fa035681-15ce-488e-859e-200bb2ca90ac"
                                    ]
                                },
                                "type": "matcher"
                            }
                        ]
                    }
                },
                {
                    "type": "matcher",
                    "definition": {
                        "key": "~timestampu",
                        "matcher": "lt",
                        "values": [2019715200]
                    }
                }
            ]
        }
    },
    "consequences": [
        {
            "id": "48181acd-22b3-edae-bc8a-447868a7df7c",
            "type": "schema",
            "detail": {
                "id": "48181acd-22b3-edae-bc8a-447868a7df7c",
                "schema": "https://ns.adobe.com/personalization/eventHistoryOperation",
                "data": {
                    "operation": "insertIfNotExists",
                    "content": {
                        "iam.eventType": "disqualify",
                        "iam.id": "a43122c4-bf19-499f-b507-087a028d1769#fa035681-15ce-488e-859e-200bb2ca90ac"
                    }
                }
            }
        }
    ]
}

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

I have signed the Adobe Open Source CLA.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

… backwards compatibility

spoorthipujariadobe · 2025-06-20T19:14:50Z

AEPCore/Tests/EventHubTests/HistoryTests/Event+EventHistoryRequestTests.swift

@@ -46,7 +46,7 @@ class Event_EventHistoryRequestTests: XCTestCase {
        let request = event.toEventHistoryRequest()

        // 6 top-level keys + 5 flattened inner dictionary keys


nit: Do we want to update this comment?

Yes, thanks for catching that! Updated

spoorthipujariadobe · 2025-06-20T19:34:33Z

AEPCore/Tests/RulesTests/EventDataFlattenerTests.swift

+            "a.b": ["c": 1],
+            "a": ["b": 2]


nit: can we also add a case where we have ["a.b" : 1 , "a": ["b": 2]] and verify the final dictionary has only one key a.b whose value is 2

I agree it would be nice to deterministically test the merge order, but my understanding is that the test case as is would implicitly rely on Swift's dictionary key ordering which is also not guaranteed, and I don't think should be relied on in a test case. Given that we also document that in the case of flattened dictionary key collision the final result is undefined I dont think we should have to test the undefined-ness portion as all we're really testing is Swift's underlying dictionary ordering implementation.

I did create a new test case to validate that the two given values result in the same flattened key though:

func testSameKeyPathFromDifferentShapes() { let dottedKey: [String: Any] = ["a.b": 1] let nestedKey: [String: Any] = ["a": ["b": 1]] XCTAssertEqualDictionaries(dottedKey.flattening(), nestedKey.flattening()) }

Please let me know what you think!

spoorthipujariadobe

Looks good! Just a few nits on extra tests

cdhoffmann

LGTM, excellent documentation. Agree with Spoorthi on the collision testing.

timkimadobe

Thanks for the review @spoorthipujariadobe and @cdhoffmann! Updated based on feedback

timkimadobe · 2025-06-24T22:55:47Z

AEPCore/Tests/EventHubTests/HistoryTests/Event+EventHistoryRequestTests.swift

@@ -46,7 +46,7 @@ class Event_EventHistoryRequestTests: XCTestCase {
        let request = event.toEventHistoryRequest()

        // 6 top-level keys + 5 flattened inner dictionary keys


Yes, thanks for catching that! Updated

timkimadobe · 2025-06-24T23:12:20Z

AEPCore/Tests/RulesTests/EventDataFlattenerTests.swift

+            "a.b": ["c": 1],
+            "a": ["b": 2]


I agree it would be nice to deterministically test the merge order, but my understanding is that the test case as is would implicitly rely on Swift's dictionary key ordering which is also not guaranteed, and I don't think should be relied on in a test case. Given that we also document that in the case of flattened dictionary key collision the final result is undefined I dont think we should have to test the undefined-ness portion as all we're really testing is Swift's underlying dictionary ordering implementation.

I did create a new test case to validate that the two given values result in the same flattened key though:

func testSameKeyPathFromDifferentShapes() { let dottedKey: [String: Any] = ["a.b": 1] let nestedKey: [String: Any] = ["a": ["b": 1]] XCTAssertEqualDictionaries(dottedKey.flattening(), nestedKey.flattening()) }

Please let me know what you think!

…s for some cases

…arity with Android

timkimadobe · 2025-06-25T00:22:40Z

AEPCore/Sources/eventhub/history/EventData+FNV1A32.swift

                    valueAsString = String(describing: (value as! Double))
                case is Bool:
                    valueAsString = String(describing: (value as! Bool))
+                case is NSNull:


Added filtering out for NSNull values in the FNV1A32 hash generation, since my understanding is their inclusion was never intended?

Hey Tim, need clarification here. What was the behavior before here? If the value was nil, what would the default case string look like?

The behavior before was when you pass in a key value pair like:

["key": NSNull()]

the value would hit the last case, the default:

default: valueAsString = String(describing: value) }

turning NSNull into "<null>" instead of being filtered out, and the string for the hash would be:
"key:<null>"

Ok gotcha. Yeah was curious about what String(describing:value) did with NSNull. Thanks for grabbing that. I'm ok with this change but just want to be sure @sbenedicadb has some input here since he wrote this i believe.

Of course! And yes, I discussed with Steve and he said:

"i'm good with adding it on the principle of keeping it consistent with the android behavior."

timkimadobe added 4 commits June 19, 2025 15:32

Add support for array flattening and add and update tests

263870d

Add ability for all_url to not use the new array flattening logic for…

361cec8

… backwards compatibility

Update examples for clarity

992628d

Apply swift formatting

96d6d8b

timkimadobe requested review from cdhoffmann, sbenedicadb and spoorthipujariadobe June 20, 2025 00:25

spoorthipujariadobe reviewed Jun 20, 2025

View reviewed changes

spoorthipujariadobe approved these changes Jun 20, 2025

View reviewed changes

cdhoffmann approved these changes Jun 23, 2025

View reviewed changes

Add dictionary shape test and update test case comment

f631950

timkimadobe commented Jun 24, 2025

View reviewed changes

timkimadobe added 2 commits June 24, 2025 16:55

Add test cases for flattenArrays false, tighten test case expectation…

8503e72

…s for some cases

Fix NSNull values filtering in the FNV1A32 hash generation and test p…

8788ad6

…arity with Android

timkimadobe commented Jun 25, 2025

View reviewed changes

timkimadobe merged commit 30f835b into adobe:feature/historical-consequence Jul 8, 2025
17 checks passed

timkimadobe deleted the rules-array-access branch July 8, 2025 21:12

This was referenced Jul 24, 2025

Support for new historical rules condition and consequences #1161

Closed

Support new historical rules condition and consequences #1162

Merged

		@@ -46,7 +46,7 @@ class Event_EventHistoryRequestTests: XCTestCase {
		let request = event.toEventHistoryRequest()

		// 6 top-level keys + 5 flattened inner dictionary keys

Conversation

timkimadobe commented Jun 20, 2025

Description

Given

Impact

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spoorthipujariadobe Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spoorthipujariadobe left a comment

Choose a reason for hiding this comment

Uh oh!

cdhoffmann left a comment

Choose a reason for hiding this comment

Uh oh!

timkimadobe left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

spoorthipujariadobe Jun 20, 2025 •

edited

Loading