Skip to content

[ML] Data frames: Updated stats structure.#42923

Merged
walterra merged 6 commits intoelastic:masterfrom
walterra:ml-df-fix-state
Aug 9, 2019
Merged

[ML] Data frames: Updated stats structure.#42923
walterra merged 6 commits intoelastic:masterfrom
walterra:ml-df-fix-state

Conversation

@walterra
Copy link
Copy Markdown
Contributor

@walterra walterra commented Aug 8, 2019 •
edited
Loading

Summary

Fixes #42825

  • Reflects the changes to transform stats on the ES side in [ML-DataFrame] Combine task_state and indexer_state in _stats elasticsearch#45276
  • Updated mock __mocks__/data_frame_transform_list_row.json to reflect the new stats structure
  • Previously there were stats attributes task_state and indexer_state, now there is only state.
  • Renamed DATA_FRAME_TASK_STATE to DATA_FRAME_TRANSFORM_STATE .
  • Extended possible states to aborting, failed, indexing, started, stopped, stopping.
  • A stopped batch transform's stats no longer returns the progress until it was stopped. Instead of falling back and showing 0% with an empty progress bar, nothing will be shown. If the job is continued and returns progress again, the progress bar will also be shown again.
  • The expanded row's details tab now exposes hard-coded stats/state attributes instead of inferring attributes dynamically from objects. This was done so we can do custom formatting for date fields for example.

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

For maintainers

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Aug 8, 2019

💚 Build Succeeded

@walterra walterra self-assigned this Aug 8, 2019
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Aug 8, 2019

Pinging @elastic/ml-ui

@walterra walterra added release_note:skip Skip the PR/issue when compiling release notes regression labels Aug 8, 2019
@walterra walterra marked this pull request as ready for review August 8, 2019 12:42
@walterra walterra requested a review from a team as a code owner August 8, 2019 12:42
peteharverson
peteharverson approved these changes Aug 8, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

alvarezmelissa87
alvarezmelissa87 reviewed Aug 8, 2019
View reviewed changes
...lugins/ml/public/data_frame/pages/transform_management/components/transform_list/actions.tsx
isPrimary: true,
render: (item: DataFrameTransformListRow) => {
if (item.stats.task_state !== DATA_FRAME_TASK_STATE.STARTED) {
if (item.stats.state === DATA_FRAME_TRANSFORM_STATE.STOPPED) {
Copy link
Copy Markdown
Contributor

@alvarezmelissa87 alvarezmelissa87 Aug 8, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unclear as to why this changed from checking STARTED to STOPPED?
Makes sense that we'd show the start action if it's stopped but just curious why it used to be STARTED - was it wrong before?

Copy link
Copy Markdown
Contributor Author

@walterra walterra Aug 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is that state now can return more different strings like indexing, stopping, aborting which all mean it's more or less still running. So I flipped the check to check against stopped.

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Aug 8, 2019

💚 Build Succeeded

alvarezmelissa87
alvarezmelissa87 approved these changes Aug 8, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@alvarezmelissa87 alvarezmelissa87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ⚡️

jgowdyelastic
jgowdyelastic approved these changes Aug 8, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@jgowdyelastic jgowdyelastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@walterra walterra merged commit 4a9300f into elastic:master Aug 9, 2019
@walterra walterra deleted the ml-df-fix-state branch August 9, 2019 07:13
@walterra walterra mentioned this pull request Aug 9, 2019
Merged
walterra added a commit to walterra/kibana that referenced this pull request Aug 9, 2019
@walterra
[ML] Data frames: Updated stats structure. (elastic#42923)
31253b2 
* Reflects the changes to transform stats on the ES side in elastic/elasticsearch#45276
* Updated mock __mocks__/data_frame_transform_list_row.json to reflect the new stats structure
* Previously there were stats attributes task_state and indexer_state, now there is only state.
* Renamed DATA_FRAME_TASK_STATE to DATA_FRAME_TRANSFORM_STATE .
* Extended possible states to aborting, failed, indexing, started, stopped, stopping.
* A stopped batch transform's stats no longer returns the progress until it was stopped. Instead of falling back and showing 0% with an empty progress bar, nothing will be shown. If the job is continued and returns progress again, the progress bar will also be shown again.
* The expanded row's details tab now exposes hard-coded stats/state attributes instead of inferring attributes dynamically from objects. This was done so we can do custom formatting for date fields for example.
walterra added a commit that referenced this pull request Aug 9, 2019
@walterra
[ML] Data frames: Updated stats structure. (#42923) (#43002)
c8548e0 
* Reflects the changes to transform stats on the ES side in elastic/elasticsearch#45276
* Updated mock __mocks__/data_frame_transform_list_row.json to reflect the new stats structure
* Previously there were stats attributes task_state and indexer_state, now there is only state.
* Renamed DATA_FRAME_TASK_STATE to DATA_FRAME_TRANSFORM_STATE .
* Extended possible states to aborting, failed, indexing, started, stopped, stopping.
* A stopped batch transform's stats no longer returns the progress until it was stopped. Instead of falling back and showing 0% with an empty progress bar, nothing will be shown. If the job is continued and returns progress again, the progress bar will also be shown again.
* The expanded row's details tab now exposes hard-coded stats/state attributes instead of inferring attributes dynamically from objects. This was done so we can do custom formatting for date fields for example.
jloleysens added a commit to jloleysens/kibana that referenced this pull request Aug 9, 2019
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into feature/rollu…
1394f75 
…p-metrics-selectall

* 'master' of github.com:elastic/kibana: (306 commits)
  [ML] Adding job overrides to the module setup endpoint (elastic#42946)
  [APM] Fix missing RUM url (elastic#42940)
  close socket timeouts without message (elastic#42456)
  Upgrade elastic/charts to 8.1.6 (elastic#42518)
  [ML] Delete old AngularJS data visualizer and refactor folders (elastic#42962)
  Add custom formatting for Date Nanos Format (elastic#42445)
  [Vega] Shim new platform - vega_fn.js -> vega_fn.js , use ExpressionFunction (elastic#42582)
  add socket.getPeerCertificate to KibanaRequest (elastic#42929)
  [Automation] ISTANBUL PRESET PATH is not working fine with constructor(private foo) (elastic#42683)
  [ML] Data frames: Updated stats structure. (elastic#42923)
  [Code] fixed the issue that the repository can not be deleted in some cases. (elastic#42841)
  [kbn-es] Support for passing regex value to ES (elastic#42651)
  Connect to Elasticsearch via SSL when starting kibana with `--ssl` (elastic#42840)
  Add Elasticsearch SSL support for integration tests (elastic#41765)
  Fix duplicate fetch in Visualize (elastic#41204)
  [DOCS] TSVB and Timelion clean up (elastic#42953)
  [Maps] [File upload] Fix maps geojson upload hanging on index step (elastic#42623)
  [APM] Use rounded bucket sizes for transaction distribution (elastic#42830)
  [yarn.lock] consistent resolve domain (elastic#42969)
  [Uptime] [Test] Repurpose unit test assertions to avoid flakiness (elastic#40650)
  ...
@walterra walterra mentioned this pull request Aug 19, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alvarezmelissa87 alvarezmelissa87 alvarezmelissa87 approved these changes

@peteharverson peteharverson peteharverson approved these changes

@jgowdyelastic jgowdyelastic jgowdyelastic approved these changes

Assignees

@walterra walterra

Labels

Feature:Transforms Transforms :ml regression release_note:skip Skip the PR/issue when compiling release notes v7.4.0 v8.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[ML] Data Frames: Update state/stats.

5 participants

@walterra @elasticmachine @alvarezmelissa87 @peteharverson @jgowdyelastic