Skip to content

[Logs UI] Add categories table to the categorization tab#53004

Merged
weltenwort merged 33 commits intoelastic:masterfrom
weltenwort:logs-ui-categorization-add-results
Jan 13, 2020
Merged

[Logs UI] Add categories table to the categorization tab#53004
weltenwort merged 33 commits intoelastic:masterfrom
weltenwort:logs-ui-categorization-add-results

Conversation

@weltenwort
Copy link
Copy Markdown
Member

@weltenwort weltenwort commented Dec 13, 2019 •
edited
Loading

Summary

This renders the log entry categories after the ML jobs have been set up previously.

closes #42776
closes #42065

Previews

grafik

grafik

Hints for testing

  • The ML job expects the index mappings to have at least
    • @timestamp (date) - the field can be overridden using the (soon to be deprecated) source settings
    • message (text)
    • event.dataset (keyword)
  • Individual documents may be missing the event.dataset, in which case their dataset will be shown as "unknown".
  • Use test data that contain more that one value for event.dataset to test the multi-select filter.
  • The test documents should span more than 10 15-minute buckets to get any useful sparklines. Ideally the data span multiple days.
  • In order to get the results page to show the "update job" callout, the job_revision in the job's custom_settings at

    kibana/x-pack/legacy/plugins/ml/server/models/data_recognizer/modules/logs_ui_categories/ml/log_entry_categories_count.json

    Line 32 in 53d1c96

    "job_revision": 0
    must be greater than that of the running job.
  • In order to get the results page to show the "reconfigure job" callout, a previously included index name pattern must have been removed from the source configuration.

@weltenwort weltenwort added Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services labels Dec 13, 2019
@weltenwort weltenwort self-assigned this Dec 13, 2019
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Dec 13, 2019

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

@weltenwort weltenwort force-pushed the logs-ui-categorization-add-results branch 11 times, most recently from 82d1281 to 298a57b Compare December 21, 2019 00:36
@elastic elastic deleted a comment from elasticmachine Dec 21, 2019
@weltenwort weltenwort force-pushed the logs-ui-categorization-add-results branch from 8892867 to 81fc33b Compare January 2, 2020 17:57
@weltenwort weltenwort force-pushed the logs-ui-categorization-add-results branch from 248d972 to 8b9474e Compare January 3, 2020 16:27
@elastic elastic deleted a comment from kibanamachine Jan 3, 2020
@weltenwort weltenwort requested a review from a team as a code owner January 8, 2020 17:13
@weltenwort weltenwort changed the title [Logs UI] Add results to categories tab [Logs UI] Add categories table to the categorization tab Jan 8, 2020
jasonrhodes
jasonrhodes reviewed Jan 9, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@jasonrhodes jasonrhodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just leaving a comment to "claim" this review. Will take a full look today.

@hbharding
Copy link
Copy Markdown
Contributor

hbharding commented Jan 9, 2020

image
Tiny thing - definitely not worth holding up the PR - but thought i'd bring it up. The green stroke of the sparkline is clipped when the value is close to 0. It appears thicker as the value increases. Is it possible to somehow add 1px of padding to the svg chart so the line doesn't clip near the edges?

katrin-freihofner
katrin-freihofner reviewed Jan 9, 2020
View reviewed changes
Copy link
Copy Markdown

@katrin-freihofner katrin-freihofner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good work!

...nfra/public/pages/logs/log_entry_categories/sections/top_categories/top_categories_table.tsx
/>
);
},
width: '220px',
Copy link
Copy Markdown

@katrin-freihofner katrin-freihofner Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
width: '220px',
width: '200px',

x-pack/legacy/plugins/infra/public/pages/logs/log_entry_rate/page_results_content.tsx
Comment on lines +268 to 272
flex: 1 0 0%;
.euiFlexGroup--responsive > .euiFlexItem {
flex-basis: auto !important;
}
Copy link
Copy Markdown

@katrin-freihofner katrin-freihofner Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, I don't get why we need this.

Copy link
Copy Markdown
Member Author

@weltenwort weltenwort Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure, @Kerry350 do you remember what this was about?

@weltenwort
Copy link
Copy Markdown
Member Author

weltenwort commented Jan 9, 2020 •
edited
Loading

Thanks for the feedback so far!

The green stroke of the sparkline is clipped when the value is close to 0. It appears thicker as the value increases. Is it possible to somehow add 1px of padding to the svg chart so the line doesn't clip near the edges?

That clipping actually happens within the elastic-charts canvas (no svg used here). I've tried to compensate by adding margins or paddings but these don't seem to affect the clipping. It's also visible in the official EUI docs, so I think it will have to be fixed upstream:

image

@jasonrhodes
Copy link
Copy Markdown
Member

jasonrhodes commented Jan 13, 2020

@elasticmachine merge upstream

@jasonrhodes
Copy link
Copy Markdown
Member

jasonrhodes commented Jan 13, 2020

This seems to be working great! I had 2 thoughts as I used it:

  1. It's weird that when you set up the job, you stay on the set up screen and the info is still editable even though it does nothing (you can still click on/off the indices, change the date range, etc). I think I expect to just be dropped onto the results screen automatically, or at least remove the irrelevant setup controls or make them read-only or some indication that the job is set up and can't continue to be edited from that screen. Not something to hold up this PR but wanted to point it out.

  2. The Analyze in ML button takes me into the ML app but I feel pretty immediately lost. Not much we can do there for now but I wonder how we make that bridge more intuitive/easy moving forward.

@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Jan 13, 2020

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@weltenwort
Copy link
Copy Markdown
Member Author

weltenwort commented Jan 13, 2020

@jasonrhodes @katrin-freihofner thanks for the reviews. Do you think this would be good to go?

@jasonrhodes good point about the form on the setup screen. I've created #54555 to track the improvement.

This was referenced Jan 13, 2020
Merged
Merged
Merged
@weltenwort weltenwort merged commit 5ef4aa1 into elastic:master Jan 13, 2020
@weltenwort weltenwort deleted the logs-ui-categorization-add-results branch January 13, 2020 19:15
@weltenwort weltenwort mentioned this pull request Jan 13, 2020
Merged
weltenwort added a commit to weltenwort/kibana that referenced this pull request Jan 13, 2020
@weltenwort
[Logs UI] Add categories table to the categorization tab (elastic#53004)
db785f2 
This renders the log entry categories after the ML jobs have been set up previously.

closes elastic#42776
closes elastic#42065
weltenwort added a commit that referenced this pull request Jan 13, 2020
@weltenwort
[Logs UI] Add categories table to the categorization tab (#53004) (#5…
fd3a8fe 
…4633)

Backports the following commits to 7.x:
 - [Logs UI] Add categories table to the categorization tab (#53004)
jkelastic pushed a commit to jkelastic/kibana that referenced this pull request Jan 17, 2020
@weltenwort @jkelastic
[Logs UI] Add categories table to the categorization tab (elastic#53004)
2ef4b35 
This renders the log entry categories after the ML jobs have been set up previously.

closes elastic#42776
closes elastic#42065
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasonrhodes jasonrhodes jasonrhodes approved these changes

+1 more reviewer

@katrin-freihofner katrin-freihofner katrin-freihofner left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@weltenwort weltenwort

Labels

Feature:Logs UI Logs UI feature release_note:enhancement Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.6.0 v8.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Logs UI] Add the log categories result screen [Logs UI] Create API route to access common log category analysis results

6 participants

@weltenwort @elasticmachine @hbharding @jasonrhodes @kibanamachine @katrin-freihofner