Skip to content

Add support for nested lists to map flattening logic#777

Merged
spoorthipujariadobe merged 9 commits intoadobe:feature/disqualificationfrom
spoorthipujariadobe:feature/disqualification
Jul 15, 2025
Merged

Add support for nested lists to map flattening logic#777
spoorthipujariadobe merged 9 commits intoadobe:feature/disqualificationfrom
spoorthipujariadobe:feature/disqualification

Conversation

@spoorthipujariadobe
Copy link
Copy Markdown
Contributor

@spoorthipujariadobe spoorthipujariadobe commented Jun 17, 2025

Description

This PR enhances the map flattening logic to add support for nested lists and arrays. The flattening logic in MapExtensions.kt has been updated to properly index list/array elements in the flattened keys and supports mixed data structures (maps containing lists, lists containing maps).

Example

Before:

{rootKey: {key1: [value0, value1]}} -> {rootKey.key1: [value0, value1]}

After:

{rootKey: {key1: [value0, value1]}} -> {rootKey.key1.0: value0, rootKey.key1.1: value1}

Impact

This change affects how event data map is flattened in:

  • Event data key lookup for rules matcher condition: Since key lookup in nested lists was never supported, this is a net new feature for this case and shouldn't create any backwards compatibility issues.
  • Event data hash generation for event history read and writes: Messaging SDK was the only extension to use event history and it never recorded maps with nested lists so this change should not have any impact.
  • Converting event data to encoded URL format using ~all_url: Since customers use this in their Launch rules with postback consequence, this change might impact their existing rules. A backwards compatible carve out for the token ~all_url` is needed so that arrays remain unflattened and preserve the existing behavior. This is achieved by implementing a flag in map flattening which controls whether or not to flatten inner arrays.

Related Issue

Motivation and Context

With the existing logic, if a map contains a nested list, it is considered as unsupported for flattening and the resulting flattened map has the list added as an object, making it impossible to lookup for an element at a specific index in the list for rules matcher condition.

We need this ability to support content card disqualification when the card is dismissed. The rule condition in this case verifies that the dismiss event happened on the specific card we want to disqualify and writes a disqualify event to event history as a consequence. To verify this card association, we need to look up and match the activity id which is present inside the propositions array in the dismiss event data.

Disqualify rule example:


{
    "condition": {
        "type": "group",
        "definition": {
            "logic": "and",
            "conditions": [
                {
                    "type": "group",
                    "definition": {
                        "logic": "and",
                        "conditions": [
                             {
                                "definition": {
                                    "key": "~type",
                                    "matcher": "eq",
                                    "values": [
                                        "com.adobe.eventType.edge"
                                    ]
                                },
                                "type": "matcher"
                            },
                            {
                                "definition": {
                                    "key": "~source",
                                    "matcher": "eq",
                                    "values": [
                                        "com.adobe.eventSource.requestContent"
                                    ]
                                },
                                "type": "matcher"
                            },
                            {
                                "definition": {
                                    "key": "xdm.eventType",
                                    "matcher": "eq",
                                    "values": [
                                        "decisioning.propositionDismiss"
                                    ]
                                },
                                "type": "matcher"
                            },
                            {
                                "definition": {
                                    "key": "xdm._experience.decisioning.propositions.0.scopeDetails.activity.id",
                                    "matcher": "eq",
                                    "values": [
                                        "a43122c4-bf19-499f-b507-087a028d1769#fa035681-15ce-488e-859e-200bb2ca90ac"
                                    ]
                                },
                                "type": "matcher"
                            }
                        ]
                    }
                },
                {
                    "type": "matcher",
                    "definition": {
                        "key": "~timestampu",
                        "matcher": "lt",
                        "values": [2019715200]
                    }
                }
            ]
        }
    },
    "consequences": [
        {
            "id": "48181acd-22b3-edae-bc8a-447868a7df7c",
            "type": "schema",
            "detail": {
                "id": "48181acd-22b3-edae-bc8a-447868a7df7c",
                "schema": "https://ns.adobe.com/personalization/eventHistoryOperation",
                "data": {
                    "operation": "insertIfNotExists",
                    "content": {
                        "iam.eventType": "disqualify",
                        "iam.id": "a43122c4-bf19-499f-b507-087a028d1769#fa035681-15ce-488e-859e-200bb2ca90ac"
                    }
                }
            }
        }
    ]
}

How Has This Been Tested?

Updated related tests in MapUtilsTests.java and LaunchTokenFinderTest.kt to reflect the new flattening behavior

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have signed the Adobe Open Source CLA.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@spoorthipujariadobe
Copy link
Copy Markdown
Contributor Author

spoorthipujariadobe commented Jun 17, 2025

Question for reviewers: Since this was also a customer requested feature, should we also document this in our public docs at https://developer.adobe.com/client-sdks/home/base/mobile-core/rules-engine/technical-details/#matching-and-retrieving-values-by-keys?

@codecov
Copy link
Copy Markdown

codecov bot commented Jun 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.39%. Comparing base (d0c72be) to head (5fc9bba).
Report is 3 commits behind head on feature/disqualification.

Additional details and impacted files
@@                      Coverage Diff                       @@
##             feature/disqualification     #777      +/-   ##
==============================================================
+ Coverage                       75.37%   75.39%   +0.02%     
- Complexity                       2338     2339       +1     
==============================================================
  Files                             217      218       +1     
  Lines                           10889    10898       +9     
  Branches                         1411     1412       +1     
==============================================================
+ Hits                             8207     8216       +9     
  Misses                           2051     2051              
  Partials                          631      631              
Flag Coverage Δ
android-functional-tests 28.17% <46.15%> (-0.01%) ⬇️
android-unit-tests 64.35% <61.54%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...al/eventhub/history/AndroidEventHistoryDatabase.kt 79.61% <100.00%> (+0.40%) ⬆️
...e/marketing/mobile/internal/util/ListExtensions.kt 100.00% <100.00%> (ø)
...be/marketing/mobile/internal/util/MapExtensions.kt 81.82% <100.00%> (ø)
...obile/launch/rulesengine/LaunchRulesConsequence.kt 90.07% <100.00%> (+0.15%) ⬆️
...ing/mobile/launch/rulesengine/LaunchTokenFinder.kt 76.36% <100.00%> (+0.44%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yangyansong-adbe
Copy link
Copy Markdown
Contributor

@spoorthipujariadobe It's better to add some functional/module tests for this new feature, such as

  • Add a real rules file that contains the array index token
  • Verify the rules engine extracts the token value correctly

flattenedMap[expandedKey] = value
}
}
is List<*>, is Array<*> -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of defining the List and Array flattening separately on their respective types?:

internal fun Map<String, Any?>.flattening(prefix: String = "", flattenArrays: Boolean = true): Map<String, Any?>
internal fun List<Any?>.flattening(prefix: String = ""): Map<String, Any?>
internal fun Array<out Any?>.flattening(prefix: String = "") = toList().flattening(prefix)

I was thinking if we ever need to flatten top level List/Array values, having it defined on their respective types keeps the implementation stable

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. Modified it

*/
@JvmSynthetic
internal fun Map<String, Any?>.flattening(prefix: String = ""): Map<String, Any?> {
internal fun Map<String, Any?>.flattening(prefix: String = "", shouldFlattenListAndArray: Boolean = true): Map<String, Any?> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nit: what do you think of renaming to something like: flattenLists?

}

@Test
fun testMapFlatteningWithDotInKey() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test case implicitly relies on Kotlin's internal implementation of map key ordering logic right? Should we update this test case to simply validate that the flattened key results are equal to the expected a.b?

)
assertEquals(expectedMap, flattenedMap)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add a base case test of empty map that is flattened == empty map?

}

@Test
fun testMapFlatteningWithListFlatteningDisabled() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add the Array value case (that is, list is tested but would be nice to validate Array is treated the same)

@spoorthipujariadobe spoorthipujariadobe merged commit 55ea8ff into adobe:feature/disqualification Jul 15, 2025
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants