Added handling for insert and insertIfNotExists rule consequence#749
Conversation
…re-android into feature/disqualification
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## feature/disqualification #749 +/- ##
===========================================================
Coverage ? 74.84%
Complexity ? 2323
===========================================================
Files ? 217
Lines ? 10789
Branches ? 1402
===========================================================
Hits ? 8075
Misses ? 2071
Partials ? 643
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Thanks Spoorthi! Sorry for the delay in reviewing
Question semi-related to this feature, but not touched in this PR:
I noticed that in AndroidEventHistory's recordEvent:
val fnv1aHash = convertMapToFnv1aHash(event.eventData, event.mask) can return -1 if event data is null, but this is not handled in the method and inserts into event history, is this the intended behavior?
In iOS, the default is 0 and the event is not recorded: https://github.com/timkimadobe/aepsdk-core-ios/blob/872b423b3d7767839b050cdfb9eced9ee6480a2c/AEPCore/Sources/eventhub/Event.swift#L65
cc @sbenedicadb
Also sorry haven't fully reviewed the test cases, will do that as I implement them in iOS
| } | ||
|
|
||
| override fun recordHistoricalEvent(event: Event, handler: EventHistoryResultHandler<Boolean>) { | ||
| EventHub.shared.eventHistory?.recordEvent(event, handler) |
There was a problem hiding this comment.
To Steve's point on my PR in iOS, should we add a call to the handler here with false if eventHistory is null? And similarly for getHistoricalEvents too: https://github.com/adobe/aepsdk-core-ios/pull/1121/files/166478db789731592a63b0e58bf4fa950b84eaf1#r2017691177
| private const val EVENT_HISTORY_OPERATION_KEY = "operation" | ||
| private const val EVENT_HISTORY_KEYS_KEY = "keys" | ||
| private const val EVENT_HISTORY_CONTENT_KEY = "content" | ||
| private const val EVENT_HISTORY_TOKEN_PREFIX = "~" |
There was a problem hiding this comment.
Forgot to remove this in mine, but with the removal of keys support, I think we can remove:
EVENT_HISTORY_TOKEN_PREFIX and EVENT_HISTORY_KEYS_KEY
| } | ||
|
|
||
| // Note `content` doesn't need to be resolved here because it was already resolved by LaunchRulesConsequence.process(event: Event, matchedRules: List<LaunchRule>) | ||
| val content = DataReader.getTypedMap(Any::class.java, schemaData, EVENT_HISTORY_CONTENT_KEY) |
There was a problem hiding this comment.
Doesn't getTypedMap throw on cast failure? The Kotlin side would need to handle any throws right?:
Just an idea but something along the lines of:
val content: Map<String, Any>?
try {
content = DataReader.getTypedMap(Any::class.java, schemaData, EVENT_HISTORY_CONTENT_KEY)
} catch (e: DataReaderException) {
Log.warning(
LaunchRulesEngineConstants.LOG_TAG,
logTag,
"Unable to process eventHistoryOperation operation for consequence ${consequence.id}, 'content' is either missing or improperly formatted in 'details.data': ${e.localizedMessage}"
)
return
}There was a problem hiding this comment.
Good catch! I will change to use optTypedMap and return null so the check on the next line causes a fast fail
| get() = detail?.get("schema") as? String | ||
|
|
||
| private val RuleConsequence.detailData: Map<String, Any>? | ||
| get() = DataReader.getTypedMap(Any::class.java, detail, "data") |
There was a problem hiding this comment.
This cast with potential throw also needs to be handled right?
There was a problem hiding this comment.
I will change it to use optTypedMap since we already have a null check for detailData
| } | ||
|
|
||
| @Test | ||
| fun testRecordHistoricalEvent() { |
There was a problem hiding this comment.
Sorry not sure if this is covered already somewhere else, but could we also add a case that tests when recording an event without a mask applied?
| val flattenedData = eventData?.flattening() ?: emptyMap() | ||
|
|
||
| // Filter the flattened data based on mask if provided | ||
| val filteredData: Map<String, Any?> = if (!mask.isNullOrEmpty()) { |
There was a problem hiding this comment.
I think this is out of scope for this PR, but I noticed that there is a logic difference between iOS and Android in general for the fnv1a32 method:
- Android checks that the mask is not null OR empty
- iOS only checks that the mask is not null: https://github.com/timkimadobe/aepsdk-core-ios/blob/872b423b3d7767839b050cdfb9eced9ee6480a2c/AEPCore/Sources/eventhub/history/EventData%2BFNV1A32.swift#L37
Looking at the history for Android changes, it has been like this for a while, even before the Kotlin conversion:
It's a breaking change to reconcile them, but should we make them consistent?
There was a problem hiding this comment.
I changed the implementation in the newly added toEventHistoryRequest extension function. Discussing offline about changing the existing fnv1a32 Map extension function since its a breaking change
There was a problem hiding this comment.
Changed the MapExtension. fnv1a32 to not write to event history if mask is empty map
| false | ||
| ) { count -> | ||
| eventCounts = count | ||
| latch.countDown() |
There was a problem hiding this comment.
Does this wrapper for querying historical conditions help here?
There was a problem hiding this comment.
Woah did not know this existed! Thanks Prashanth
There was a problem hiding this comment.
Same, thanks! Will replace it
On closer inspection the linked wrapper function has a limitation where in case of a database look up exception, the method returns 0 instead of throwing the exception. The insertIfNotExists logic would understand the 0 as event not existing in history and go ahead with the insertion, which is the opposite of what we want. Will leave the logic as is and create a Github issue to fix this wrapper function
| * @param to the end time that represents the upper bounds of the date range used when looking up an Event | ||
| * @return an [EventHistoryRequest] with mask derived from this event's data | ||
| */ | ||
| internal fun Event.toEventHistoryRequest( |
There was a problem hiding this comment.
Can this be moved to com.adobe.marketing.mobile.internal.util as EventExtensions is this only being used internally.
| if (StringUtils.isNullOrEmpty(consequence.detailId) || | ||
| StringUtils.isNullOrEmpty(consequence.schema) || |
There was a problem hiding this comment.
nit: consequence.detailId.isNullOrEmpty() .. , as we have been using kotlin extensions where available.
| .chainToParentEvent(parentEvent) | ||
| .build() | ||
|
|
||
| when (operation) { |
There was a problem hiding this comment.
Since this when statement has only 2 clauses, one of which is else - transitioning to the following structure will be easier to follow.
if (!(CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT || CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT_IF_NOT_EXISTS))) {
return;
}
if (CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT_IF_NOT_EXISTS) {
// do some pre-reqs
}
// rest of your processing
There was a problem hiding this comment.
I also implemented this as a switch in the iOS implementation with the intention of making any future cases easy to add on - would you say that this if structure handles the current case better? Would a switch be preferable if there were actually more cases? Or ifs would still be preferable?
There was a problem hiding this comment.
Generally I see it this way -
- Try and not nest where possible.
whenandswitchblocks are preferable for maintainability if there are multiple mutually exclusive cases.- In case there is just one clause use a simple if/else for readability.
In this particular code, two cases are combined within a when, but still we still need an if inside to distinguish one, causing nesting on the condition on which the when was applied on.
So switching this to an early return pattern seemed cleaner because it reads linearly - Not interested in anything other than X and Y. If X, do something first. Then do shared processing for X and Y.
If we were to add more such cases, we can think of having a pre-processing step as a separate method which takes in the operation, goes through the when block on it and decides what has to be done before passing this off to actual processing.
fun handleOperation() {
preprocess(operation)
process(operation)
}
fun preprocess(operation) {
when(operation) {
L,M -> { common flow for L,M }
X -> { }
Y -> { }
else -> { }
}
}
fun process(operation) {
when(operation) {
L -> { }
M -> { }
X -> { }
Y -> { }
else -> { }
}
}
There was a problem hiding this comment.
For sake of readability I changed it to use if-else for now
There was a problem hiding this comment.
One question - instead of a outer negated OR, what if we use not equals ANDed together?
so instead of:
if (!(CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT || CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT_IF_NOT_EXISTS)))
if (!CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT && !CONSEQUENCE_EVENT_HISTORY_OPERATION_INSERT_IF_NOT_EXISTS)))
I feel like the intent could be more intuitive when read?
| ) | ||
| return | ||
| } | ||
| when (consequence.schema) { |
There was a problem hiding this comment.
Recommend switching to if/else
Discussed offline and changed the method |
| consequence.schema.isNullOrBlank() || | ||
| consequence.detailData.isNullOrEmpty() | ||
| ) { | ||
| Log.error( |
There was a problem hiding this comment.
Sorry I missed this earlier, but I believe Steve recommended changing these to warning instead of error: https://github.com/adobe/aepsdk-core-ios/pull/1121/files/f0d428b1e99eed5972634b0ab72c68bd8de7a801#r2017715694
| */ | ||
| private fun processEventHistoryOperation(consequence: RuleConsequence, parentEvent: Event) { | ||
| val schemaData = consequence.detailData ?: run { | ||
| Log.error( |
There was a problem hiding this comment.
Warning level here too
| if (result) { | ||
| extensionApi.dispatch(eventToRecord) | ||
| } else { | ||
| Log.trace( |
There was a problem hiding this comment.
Should this be warning level?
| Log.warning( | ||
| LaunchRulesEngineConstants.LOG_TAG, | ||
| logTag, | ||
| "Event History operation for id ${consequence.id} - Unable to process 'insertIfNotExists' operation, event hash is 0}" |
There was a problem hiding this comment.
nit: i think theres a typo } at the end of the log string?
timkimadobe
left a comment
There was a problem hiding this comment.
Sorry for the delay in reviewing the test cases Spoorthi! Updated with some feedback
| // "data": { | ||
| // "operation": "insert", | ||
| // "content": { | ||
| // "key1": "value1" |
There was a problem hiding this comment.
nit: I know it doesn't affect what is being validated but the actual JSON file has:
"content": {
"key1": "value1",
"key2": "value2"
}
🙈
| if (consequence.detailId.isNullOrBlank() || | ||
| consequence.schema.isNullOrBlank() || |
There was a problem hiding this comment.
After going through the test cases, I believe this should be only a null check?
The spec for the rules consequence schema type only specifies that it must be a valid String
And in both iOS and Android Messaging, empty string values for both id and schema are also not interpreted as error cases:
iOS
static func fromRuleConsequenceEvent(_ event: Event) -> PropositionItem? {
guard let id = event.schemaId, let schema = event.schemaType, let schemaData = event.schemaData else {
return nil
}
return PropositionItem(itemId: id, schema: schema, itemData: schemaData)
}var schemaId: String? {
details?[MessagingConstants.Event.Data.Key.ID] as? String
}
var schemaType: SchemaType? {
guard let schemaString = details?[MessagingConstants.Event.Data.Key.SCHEMA] as? String else {
return nil
}
return SchemaType(from: schemaString)
}Android
static PropositionItem fromRuleConsequenceDetail(final Map<String, Object> consequenceDetail) {
PropositionItem propositionItem = null;
try {
final String uniqueId =
DataReader.getString(
consequenceDetail, MessagingConstants.ConsequenceDetailKeys.ID);
final SchemaType schema =
SchemaType.fromString(
DataReader.getString(
consequenceDetail,
MessagingConstants.ConsequenceDetailKeys.SCHEMA));
...| private fun processSchemaConsequence(consequence: RuleConsequence, parentEvent: Event) { | ||
| if (consequence.detailId.isNullOrBlank() || | ||
| consequence.schema.isNullOrBlank() || | ||
| consequence.detailData.isNullOrEmpty() |
There was a problem hiding this comment.
I believe the spec also mentions that the detail data is allowed to be empty - should we move this validation to the specific schema handler logic (in this case under processEventHistoryOperation)?
| } | ||
|
|
||
| @Test | ||
| fun `Test Schema Event History Insert Operation When detail id Is Empty`() { |
There was a problem hiding this comment.
If the implementation is updated to accept empty string for id, can we update this test to pass instead?
| }, | ||
| "consequences": [ | ||
| { | ||
| "type": "schema", |
There was a problem hiding this comment.
This consequence is missing the top level id field and I believe causes the test to fail at the consequence construction stage, not the detail -> id check stage
| } | ||
|
|
||
| @Test | ||
| fun `Test Schema Event History Insert Operation When detail id Is Null`() { |
There was a problem hiding this comment.
Can we also create a test case that covers when the detail "id" = null case?
| } | ||
|
|
||
| @Test | ||
| fun `Test Schema Event History Insert Operation When detail schema Is Null`() { |
There was a problem hiding this comment.
Same thing for schema, can we create another test case that validate the "schema" = null case?
| } | ||
|
|
||
| @Test | ||
| fun `Test Schema Event History Insert Operation When detail data Is Null`() { |
There was a problem hiding this comment.
Same thing for data, can we create another test case that validate the "data" = null case?
| } | ||
|
|
||
| @Test | ||
| fun `Test Schema Event History Insert Operation When Operation Is Null`() { |
There was a problem hiding this comment.
Could we group these cases with the other missing or null property check cases? So overall the flow could be like:
- Property validation checks
- Schema checks
- Schema data checks
- Schema data property checks
insertlogical checksinsertIfNotExistslogical checks- Event History failure checks
And can we also add a case for "operation" = null?
| } | ||
|
|
||
| @Test | ||
| fun `Test Schema Event History Insert Operation When Content Is Null`() { |
There was a problem hiding this comment.
Can we add a case for "content" = null?
| val eventHash = eventToRecord.eventData.fnv1a32() | ||
| if (eventHash == 0L) { | ||
| Log.warning( | ||
| LaunchRulesEngineConstants.LOG_TAG, | ||
| logTag, | ||
| "Event History operation for id ${consequence.id} - Unable to process 'insertIfNotExists' operation, event hash is 0" |
There was a problem hiding this comment.
Can we move this check to before the if check for insertIfNotExists? Since this requirement exists for both cases?
| ) | ||
| return | ||
| } | ||
| if (eventCounts >= 1) { |
There was a problem hiding this comment.
Also on second thought for this condition, do we also want to check that the eventCounts is also not -1? Since on database error, AndroidEventHistory getEvents sets count to -1 in that case?
I feel like it might be better to handle the database error case by not trying to record than to record?
There was a problem hiding this comment.
Good catch! I've added the check as well as all the tests and comments mentioned above
b197f79
into
adobe:feature/disqualification
Description
This PR implements support for the new
insertandinsertIfNotExistsconsequence support in the Core RulesEngine, leveraging the existing rules condition evaluation logic to trigger these consequences.The consequence handling has been updated to handle the new
schemaconsequence type, andhttps://ns.adobe.com/personalization/eventHistoryOperationschema value.At a high level, the consequence object has two high level properties:
Determines the type of database operation to be performed on Event History by the SDK.
- Insert: Inserts a record in the Event History Database.
- Insert if not exists: Inserts a record in the Event History Database only if there is not already a matching record with a matching hash value.
Provided key-value pairs will be added to the string used to generate the event history hash.
If the value represents a token that the SDK knows how to resolve, the resolved value will be used (such as shared state values, ~timestampu, etc.).
The primary logic for this new support is handled in the new
LaunchRulesConsequenceprocessEventHistoryOperationmethod:operationexists (which could be eitherinsertorinsertIfNotExists, but does not strictly require these two values for future expansion)contentcan have values which are tokens. Token values are reformatted and resolved using theLaunchTokenFinderbefore the consequence processing begins.content, including replaced token values, as its event data. The event mask is set up with all of the keys fromcontent.insertIfNotExists, first check that the hash for the event does not already existtoEventHistoryRequesthas been created in to support easily convert the event data toEventHistoryRequestby flattening the event data map and extracting all the key value pairs from it, as is required by the existinggetEventsAPI's input format requirementRelated Issue
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: