[7.x] [ML] Process delimited files like semi-structured text (#56038)#56156
Merged
droberts195 merged 1 commit intoelastic:7.xfrom Jan 28, 2020
droberts195:use_csv_processor_in_file_structure_finder_ingest_7x
Merged
[7.x] [ML] Process delimited files like semi-structured text (#56038)#56156droberts195 merged 1 commit intoelastic:7.xfrom droberts195:use_csv_processor_in_file_structure_finder_ingest_7x
droberts195 merged 1 commit intoelastic:7.xfrom
droberts195:use_csv_processor_in_file_structure_finder_ingest_7x
Conversation
Changes the file upload functionality to process delimited files by splitting them into to messages, then sending these to the ingest pipeline as a single field for further processing in Elasticsearch. The csv_importer has been removed and the old sst_importer replaced with a similar message_importer that has been enhanced to cover the edge cases required by delimited file processing. Previously the file upload functionality parsed CSV in the browser, but by parsing CSV in the ingest pipeline it makes the Kibana file upload functionality more easily interchangable with Filebeat such that the configurations it creates can more easily be used to import data with the same structure repeatedly in production. Companion to elastic/elasticsearch#51492
Contributor
|
Pinging @elastic/ml-ui (:ml) |
Contributor
💚 Build SucceededTo update your PR or re-run it, just comment with: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backports the following commits to 7.x: