Skip to content

[Ingest pipelines] add support for fingerprint processor#100541

Merged
sabarasaba merged 9 commits intoelastic:masterfrom
sabarasaba:add_fingerprint_processor
May 26, 2021
Merged

[Ingest pipelines] add support for fingerprint processor#100541
sabarasaba merged 9 commits intoelastic:masterfrom
sabarasaba:add_fingerprint_processor

Conversation

@sabarasaba
Copy link
Copy Markdown
Member

@sabarasaba sabarasaba commented May 25, 2021

Fixes: #90708

Release Note

The Ingest Node Pipelines UI added support to configure a fingerprint processor. This processor computes a hash based on certain fields of the document’s content.

Default description

Screenshot 2021-05-25 at 13 41 35

New processor fields

Screenshot 2021-05-25 at 13 42 19

@sabarasaba sabarasaba added v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// Feature:Ingest Node Pipelines Ingest node pipelines management v7.14.0 release_note:feature Makes this part of the condensed release notes labels May 25, 2021
@sabarasaba sabarasaba self-assigned this May 25, 2021
@sabarasaba sabarasaba requested a review from a team as a code owner May 25, 2021 11:46
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 25, 2021

Pinging @elastic/es-ui (Team:Elasticsearch UI)

@jrodewig jrodewig self-requested a review May 25, 2021 11:52
jrodewig
jrodewig suggested changes May 25, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@jrodewig jrodewig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this @sabarasaba.

The help text for Target field and Ignore missing require some changes. I also left some minor suggestions for a few other fields. After those are addressed, I'll take another look.

jrodewig
jrodewig approved these changes May 25, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@jrodewig jrodewig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks again @sabarasaba.

alisonelizabeth
alisonelizabeth reviewed May 25, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding support for the fingerprint processor! Looks great. I left a couple minor comments in the code. Let me know what you think.

@sabarasaba
Copy link
Copy Markdown
Member Author

sabarasaba commented May 26, 2021

@elasticmachine merge upstream

alisonelizabeth
alisonelizabeth approved these changes May 26, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes @sabarasaba! I left another comment around the implementation for the method default value. Let me know what you think. I'm going to go ahead and approve to unblock.

.../application/components/pipeline_editor/components/processor_form/processors/fingerprint.tsx Outdated
method: {
type: FIELD_TYPES.TEXT,
defaultValue: 'SHA-1',
serializer: from.emptyStringToUndefined,
Copy link
Copy Markdown
Contributor

@alisonelizabeth alisonelizabeth May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the serializer will need to be updated as well. If you take a look at the serializer for sort processor/order parameter linked to above, you'll see we're stripping the default value from the request. I think we should follow the same approach here to be consistent.

Copy link
Copy Markdown
Member Author

@sabarasaba sabarasaba May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uhh, thanks for having a look again @alisonelizabeth! 🙏🏼🙏🏼🙏🏼

@sabarasaba
Copy link
Copy Markdown
Member Author

sabarasaba commented May 26, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented May 26, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
ingestPipelines 493 494 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ingestPipelines 682.5KB 686.4KB +3.9KB
Unknown metric groups

References to deprecated APIs

id before after diff
canvas 29 25 -4
crossClusterReplication 8 6 -2
fleet 22 20 -2
globalSearch 4 2 -2
indexManagement 12 7 -5
infra 261 149 -112
lens 67 45 -22
licensing 18 15 -3
lists 239 236 -3
maps 286 208 -78
ml 121 115 -6
monitoring 109 56 -53
securitySolution 386 342 -44
stackAlerts 101 95 -6
total -342

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @sabarasaba

@sabarasaba sabarasaba merged commit ae63fb1 into elastic:master May 26, 2021
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 26, 2021
@gmmorris
Merge branch 'master' into task-manager/capacity-estimation
4417858 
* master: (60 commits)
  [Docs] Index patterns REST API docs (elastic#100549)
  [Ingest pipelines] add support for fingerprint processor (elastic#100541)
  ping Core team when renovate bot bumps es client version (elastic#100662)
  [Maps] Add draw wizard (elastic#100278)
  Use documentation link service for index pattern field editor (elastic#100609)
  [Maps] filter dashboard by map extent (elastic#99860)
  [ftr] migrate screenshots and snapshots services to FtrService class (elastic#100514)
  fix anomaly functional test (elastic#100504)
  update breaking changes template to incorporate ES deprecations (elastic#100621)
  improve default time ranges (elastic#100536)
  [Gauge] Fixes wrong translations on ranges less than symbol (elastic#100535)
  [ftr] migrate "globalNav" service to FtrService class (elastic#100604)
  [ftr] migrate "testSubjects" to FtrService class (elastic#100512)
  Fix spaces test flakyness (elastic#100605)
  [Ingest pipelines] add support for ip type in convert processor (elastic#100531)
  [ftr] migrate "browser" to FtrService class (elastic#100507)
  [ftr] migrate "find" service to FtrService class (elastic#100509)
  [telemetry] report config deprecations (elastic#99887)
  [ftr] migrate "docTable" service to FtrService class (elastic#100595)
  [ftr] migrate "listingTable" service to FtrService class (elastic#100606)
  ...
sabarasaba added a commit to sabarasaba/kibana that referenced this pull request May 26, 2021
@sabarasaba
[Ingest pipelines] add support for fingerprint processor (elastic#100541
cb79de6 
)
@sabarasaba sabarasaba mentioned this pull request May 26, 2021
Merged
sabarasaba added a commit that referenced this pull request May 26, 2021
@sabarasaba
[Ingest pipelines] add support for fingerprint processor (#100541) (#…
20dca6d 
…100711)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alisonelizabeth alisonelizabeth alisonelizabeth approved these changes

+1 more reviewer

@jrodewig jrodewig jrodewig approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@sabarasaba sabarasaba

Labels

Feature:Ingest Node Pipelines Ingest node pipelines management release_note:feature Makes this part of the condensed release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// v7.14.0 v8.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add support for fingerprint ingest processor

5 participants

@sabarasaba @elasticmachine @kibanamachine @alisonelizabeth @jrodewig