[Ingest Pipelines] Load from json by jloleysens · Pull Request #70297 · elastic/kibana

jloleysens · 2020-06-30T09:11:25Z

Summary

Added basic load from JSON functionality similar to what mappings editor offers.

How to test

Go to ingest pipelines app in the Kibana management section
Create or edit an existing pipeline
Click on the Load JSON button
Paste a valid pipeline into the code editor, something like:

{
      processors: [{ set: { field: 'test', value: 123 } }],
      on_failure: [
        {
          gsub: {
            field: '_index',
            pattern: '(.monitoring-\\w+-)6(-.+)',
            replacement: '$17$2',
          },
        },
      ],
    }

And click Load and overwrite.

Notes

This functionality only does basic check to see wether we can properly deserialise the pipeline object. We still ultimately rely on ES to validate the pipeline processors for us.

Screenshots

New load json button

load json modal

error message about invalid pipeline object

Invalid json error state, load button disabled

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

- refactored the pipeline processsors editor components for portability - added CIT for load from json component

elasticmachine · 2020-06-30T09:11:28Z

Pinging @elastic/es-ui (Team:Elasticsearch UI)

jloleysens · 2020-06-30T09:17:21Z

@elasticmachine merge upstream

sebelga

Great job @jloleysens ! Tested locally and works as expected. I left a few comments in the code, no real blockers.

I had a few comments on the UX, not related to the changes on this PR but leaving them here anyways 😊

When clicking the expand on the JSON, there is no way to go back to normal view, unless we know we can hit ESCAPE. I am not sure I see the value to be able to go fullscreen.

Flyout to edit: I personally think it's strange to have the JSON editor in white background and prefer the gray background that we have everywhere. Also, I would prefer to use the FlyoutFooter bar for the actions ("Cancel" + "Update"), like the mappings editor.

Do you think we could reduce the height of the JSON editor so we can see the "Invalid JSON" error below it? As you can see here, the JSON is invalid, but I don't see it and the button is grayed out, and unless you scroll you don't know why.

I think it would be better to limit the width of the flyout as we do in the mappings editor.

x-pack/plugins/ingest_pipelines/public/application/components/pipeline_form/pipeline_form.tsx

...lugins/ingest_pipelines/public/application/components/pipeline_form/pipeline_form_fields.tsx

...plication/components/pipeline_processors_editor/components/load_from_json/modal_provider.tsx

jloleysens · 2020-07-02T12:26:42Z

@elasticmachine merge upstream

- Update form schema and form schema types - simplify the save handler - refactor processors_title to processors_header

jloleysens · 2020-07-02T16:07:51Z

@elasticmachine merge upstream

jloleysens · 2020-07-02T17:51:57Z

@elasticmachine merge upstream

jloleysens · 2020-07-03T07:07:19Z

Hm, the fleet functional test keeps failing. That or another, unrelated functional test (lens). Retrying...

jloleysens · 2020-07-03T07:07:27Z

@elasticmachine merge upstream

kibanamachine · 2020-07-03T08:42:08Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 0214f90

Build metrics

✅ unchanged

History

💔 Build #58338 failed 5b0de8b
💔 Build #58284 failed c038f8a
💔 Build #58254 failed c0c4524
💚 Build #58226 succeeded 8c38bbe
💚 Build #57455 succeeded 9a52744

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* master: (32 commits) [Ingest Pipelines] Load from json (elastic#70297) [Rum Dashbaord] Rum selected service view (elastic#70579) [Uptime] Prevent duplicate requests on load for index status (elastic#70585) [ML] Changing shared module setup function parameters (elastic#70589) [Ingest Manager] Add ability to sort to agent configs and package configs (elastic#70676) [Alerting] document requirements for developing new action types (elastic#69164) Fixed adding an extra space character on selecting alert variable in action text fields (elastic#70028) [Maps] show vector tile labels on top (elastic#69444) chore(NA): upgrade to lodash@4 (elastic#69868) Add Snapshot Restore README with quick-testing steps. (elastic#70494) [EPM] Use higher priority than default templates (elastic#70640) [Maps] Fix cannot select Solid fill-color when removing fields (elastic#70621) [kbn/optimizer] only build specified themes (elastic#70389) Fix saved query modal overlay (elastic#68826) Update component templates list to render empty prompt inside of content container. Show detail panel when deep-linked, even if there are no component templates. (elastic#70633) [Security Solution] Renames the `Investigate in Resolver` Timeline action (elastic#70634) fix 400 error on initial signals search (elastic#70618) [Maps] fix unable to edit heatmap metric (elastic#70606) Update network idle timeout (elastic#70629) [APM] Disable flaky useFetcher test (elastic#70638) ...

* master: (199 commits) [Telemetry] Add documentation about Application Usage (elastic#70624) [Ingest Manager] Improve agent unenrollment with unenroll action (elastic#70031) Handle timeouts on creating templates (elastic#70635) [Lens] Add ability to set colors for y-axis series (elastic#70311) [Uptime] Use elastic charts donut (elastic#70364) [Ingest Manager] Update registry URL to point to snapshot registry (elastic#70687) [Composable template] Create / Edit wizard (elastic#70220) [APM] Optimize services overview (elastic#69648) [Ingest Pipelines] Load from json (elastic#70297) [Rum Dashbaord] Rum selected service view (elastic#70579) [Uptime] Prevent duplicate requests on load for index status (elastic#70585) [ML] Changing shared module setup function parameters (elastic#70589) [Ingest Manager] Add ability to sort to agent configs and package configs (elastic#70676) [Alerting] document requirements for developing new action types (elastic#69164) Fixed adding an extra space character on selecting alert variable in action text fields (elastic#70028) [Maps] show vector tile labels on top (elastic#69444) chore(NA): upgrade to lodash@4 (elastic#69868) Add Snapshot Restore README with quick-testing steps. (elastic#70494) [EPM] Use higher priority than default templates (elastic#70640) [Maps] Fix cannot select Solid fill-color when removing fields (elastic#70621) ...

* WiP load from json modal ready, need to refactor more stuff * First iteration of load from JSON functionality - refactored the pipeline processsors editor components for portability - added CIT for load from json component * added comment * update deserialize with tests and make it more fault tolerant * use flyout footer * remove console.error and make the json editor a lot shorter * address PR feedback - Update form schema and form schema types - simplify the save handler - refactor processors_title to processors_header * remove unused translations Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* WiP load from json modal ready, need to refactor more stuff * First iteration of load from JSON functionality - refactored the pipeline processsors editor components for portability - added CIT for load from json component * added comment * update deserialize with tests and make it more fault tolerant * use flyout footer * remove console.error and make the json editor a lot shorter * address PR feedback - Update form schema and form schema types - simplify the save handler - refactor processors_title to processors_header * remove unused translations Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

jloleysens added 2 commits June 29, 2020 14:41

WiP load from json modal ready, need to refactor more stuff

2378751

First iteration of load from JSON functionality

34f9f0e

- refactored the pipeline processsors editor components for portability - added CIT for load from json component

jloleysens added v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// release_note:skip Skip the PR/issue when compiling release notes v7.9.0 Feature:Ingest Node Pipelines Ingest node pipelines management labels Jun 30, 2020

jloleysens requested a review from sebelga June 30, 2020 09:11

jloleysens requested a review from a team as a code owner June 30, 2020 09:11

elasticmachine and others added 2 commits June 30, 2020 03:17

Merge branch 'master' into load_from_json

dd5b02e

added comment

c4c88db

jloleysens changed the title ~~Load from json~~ [Ingest Pipelines] Load from json Jun 30, 2020

update deserialize with tests and make it more fault tolerant

9a52744

sebelga approved these changes Jul 1, 2020

View reviewed changes

elasticmachine and others added 5 commits July 2, 2020 06:26

Merge branch 'master' into load_from_json

8c38bbe

use flyout footer

4bb9f72

remove console.error and make the json editor a lot shorter

1284555

address PR feedback

c0c4524

- Update form schema and form schema types - simplify the save handler - refactor processors_title to processors_header

remove unused translations

c038f8a

Merge branch 'master' into load_from_json

5b0de8b

Merge branch 'master' into load_from_json

292f4bc

Merge branch 'master' into load_from_json

0214f90

jloleysens merged commit 5159635 into elastic:master Jul 3, 2020

jloleysens deleted the load_from_json branch July 3, 2020 08:43

jloleysens mentioned this pull request Jul 4, 2020

[7.x] [Ingest Pipelines] Load from json (#70297) #70751

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Ingest Pipelines] Load from json#70297

[Ingest Pipelines] Load from json#70297
jloleysens merged 13 commits intoelastic:masterfrom
jloleysens:load_from_json

jloleysens commented Jun 30, 2020 •

edited

Loading

Uh oh!

elasticmachine commented Jun 30, 2020

Uh oh!

jloleysens commented Jun 30, 2020

Uh oh!

sebelga left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jloleysens commented Jul 2, 2020

Uh oh!

jloleysens commented Jul 2, 2020

Uh oh!

jloleysens commented Jul 2, 2020

Uh oh!

jloleysens commented Jul 3, 2020

Uh oh!

jloleysens commented Jul 3, 2020

Uh oh!

kibanamachine commented Jul 3, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jloleysens commented Jun 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test

Notes

Screenshots

Checklist

Uh oh!

elasticmachine commented Jun 30, 2020

Uh oh!

jloleysens commented Jun 30, 2020

Uh oh!

sebelga left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jloleysens commented Jul 2, 2020

Uh oh!

jloleysens commented Jul 2, 2020

Uh oh!

jloleysens commented Jul 2, 2020

Uh oh!

jloleysens commented Jul 3, 2020

Uh oh!

jloleysens commented Jul 3, 2020

Uh oh!

kibanamachine commented Jul 3, 2020

💚 Build Succeeded

Build metrics

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jloleysens commented Jun 30, 2020 •

edited

Loading