[Azure Logs] Replace foreach+set combo with a script processor#34478
Merged
zmoog merged 2 commits intoelastic:mainfrom Feb 7, 2023
Merged
Conversation
Contributor
|
This pull request does not have a backport label.
To fixup this pull request, you need to add the backport labels for the needed
|
Contributor
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
💚 Flaky test reportTests succeeded. 🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
Contributor
Author
|
@andrewkroh, the test file test-non-interactive-user-signin.log contains the key However, the "expected" file test-non-interactive-user-signin.log-expected.json content looks correct. I guess the issue is not evident because the document uses "dot notation" for the fields. WDYT? The integrations repo stores the expected files as a plain JSON document, so this behavior is more visible. |
The `set` processor expands the dots contained in the field name into
subfield.
Sometimes attributes contained in `authentication_processing_details`
have dots, for example:
# source
{"key": "a.b.c", "value": true}
In such cases, the `set` processor would turn it into:
# this is a side-effect
{
"a": {
"b": {
"c": true
}
}
}
Instead of:
# this is the expected result
{"a.b.c": True}
b9fe22e to
615edbe
Compare
girodav
approved these changes
Feb 7, 2023
mergify bot
pushed a commit
that referenced
this pull request
Feb 7, 2023
* Replace foreach+set combo with a script processor
The `set` processor expands the dots contained in the field name into
subfield.
Sometimes attributes contained in `authentication_processing_details`
have dots, for example:
# source
{"key": "a.b.c", "value": true}
In such cases, the `set` processor would turn it into:
# this is a side-effect
{
"a": {
"b": {
"c": true
}
}
}
Instead of:
# this is the expected result
{"a.b.c": True}
* Update changelog
(cherry picked from commit a0b2db2)
mergify bot
pushed a commit
that referenced
this pull request
Feb 7, 2023
* Replace foreach+set combo with a script processor
The `set` processor expands the dots contained in the field name into
subfield.
Sometimes attributes contained in `authentication_processing_details`
have dots, for example:
# source
{"key": "a.b.c", "value": true}
In such cases, the `set` processor would turn it into:
# this is a side-effect
{
"a": {
"b": {
"c": true
}
}
}
Instead of:
# this is the expected result
{"a.b.c": True}
* Update changelog
(cherry picked from commit a0b2db2)
zmoog
added a commit
that referenced
this pull request
Feb 7, 2023
… (#34503) * Replace foreach+set combo with a script processor The `set` processor expands the dots contained in the field name into subfield. Sometimes attributes contained in `authentication_processing_details` have dots, for example: # source {"key": "a.b.c", "value": true} In such cases, the `set` processor would turn it into: # this is a side-effect { "a": { "b": { "c": true } } } Instead of: # this is the expected result {"a.b.c": True} * Update changelog (cherry picked from commit a0b2db2) Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>
zmoog
added a commit
that referenced
this pull request
Feb 8, 2023
… script processor (#34502) * [Azure Logs] Replace foreach+set combo with a script processor (#34478) * Replace foreach+set combo with a script processor The `set` processor expands the dots contained in the field name into subfield. Sometimes attributes contained in `authentication_processing_details` have dots, for example: # source {"key": "a.b.c", "value": true} In such cases, the `set` processor would turn it into: # this is a side-effect { "a": { "b": { "c": true } } } Instead of: # this is the expected result {"a.b.c": True} * Update changelog (cherry picked from commit a0b2db2) * Remove unrelated CHANGELOG entries --------- Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>
chrisberkhout
pushed a commit
that referenced
this pull request
Jun 1, 2023
* Replace foreach+set combo with a script processor
The `set` processor expands the dots contained in the field name into
subfield.
Sometimes attributes contained in `authentication_processing_details`
have dots, for example:
# source
{"key": "a.b.c", "value": true}
In such cases, the `set` processor would turn it into:
# this is a side-effect
{
"a": {
"b": {
"c": true
}
}
}
Instead of:
# this is the expected result
{"a.b.c": True}
* Update changelog
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Replace the
foreach+setprocessors combo with a singlescriptprocessor in the ingest pipeline for Azure AD sign-in logs.Why is it important?
The
setprocessor expands the dots contained in the field name into subfields.Sometimes attributes contained in
authentication_processing_detailshave dots, for example:In such cases, the
setprocessor would turn it into:Instead of:
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Related issues