[ Security Solution ] Fix Refetch logic with new timeline batching#205893
[ Security Solution ] Fix Refetch logic with new timeline batching#205893logeekal merged 7 commits intoelastic:mainfrom
Conversation
|
Pinging @elastic/security-threat-hunting-investigations (Team:Threat Hunting:Investigations) |
|
Hmmmm there is something fishy happening. When I play with adding columns, removing columns and refreshing, sometimes it resets back to the first page but keeps 10 pages of 100 elements, sometimes it stays on the same page... Screen.Recording.2025-01-13.at.5.57.20.PM.mov |
|
@michaelolo24 , @PhilippeOberti If we consider below use case:
Reseting batching to initial position ( similar to what discover does ) will not be a good UX. I think we should fetch all the batches if it goes against Discover. I have been to and fro on this. What do you think? |
I go back and forth on this as well... On one hand, I agree with you that the UX you're suggesting seems better. On the other hand though, if it is truly inevitable that Discover is going to replace timeline, is there a good enough value for us to deviate from their logic? That will only frustrate users in the future when the logic changes... It might be better to frustrate them now 😆 |
|
As per our discussion and ADR, i have updated the refetch logic to always fetch only the initial batch. Please review |
michaelolo24
left a comment
There was a problem hiding this comment.
The changes look great! If you can just validate the alert status change behavior with product beforehand, that would be helpful, but I think the default right now is sensible, and that status change actions will more likely than not take place on the alerts table rather than in timeline. 👍🏾
|
Starting backport for target branches: 8.16, 8.17, 8.18, 8.9, 8.x, 9.0 |
💚 Build Succeeded
Metrics [docs]Async chunks
History
|
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios (cherry picked from commit 54b4fac)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
…ing (#205893) (#209916) # Backport This will backport the following commits from `main` to `9.0`: - [[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)](#205893) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jatin Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-02-05T21:12:38Z","message":"[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)\n\n## Summary\n\nPR : #204034 fixed some issues\nwith timeline batching. It was not able to fix one of the issue with\n`Refetch` logic which exists in `main` ( resulting in a flaky test ) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n## Issue Description\n\nThere are 2 issues with below video:\n\n1. When user updates a status of an alert, the `Refetch` only happens on\nthe first `batch`. This behaviour is flaky currently. Even if the user\nis on nth batch, table will fetch 0th batch and reset the user's page\nback to 1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n \n\n\n3. When user clicks `Refresh` manually, then also only first (0th)\n`batch` is fetched, which should have rather fetched all the present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport","release_note:skip","Team:Threat Hunting:Investigations","v8.9.1","v9.1.0"],"title":"[ Security Solution ] Fix Refetch logic with new timeline batching","number":205893,"url":"https://github.com/elastic/kibana/pull/205893","mergeCommit":{"message":"[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)\n\n## Summary\n\nPR : #204034 fixed some issues\nwith timeline batching. It was not able to fix one of the issue with\n`Refetch` logic which exists in `main` ( resulting in a flaky test ) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n## Issue Description\n\nThere are 2 issues with below video:\n\n1. When user updates a status of an alert, the `Refetch` only happens on\nthe first `batch`. This behaviour is flaky currently. Even if the user\nis on nth batch, table will fetch 0th batch and reset the user's page\nback to 1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n \n\n\n3. When user clicks `Refresh` manually, then also only first (0th)\n`batch` is fetched, which should have rather fetched all the present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b"}},"sourceBranch":"main","suggestedTargetBranches":["8.9"],"targetPullRequestStates":[{"branch":"8.9","label":"v8.9.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205893","number":205893,"mergeCommit":{"message":"[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)\n\n## Summary\n\nPR : #204034 fixed some issues\nwith timeline batching. It was not able to fix one of the issue with\n`Refetch` logic which exists in `main` ( resulting in a flaky test ) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n## Issue Description\n\nThere are 2 issues with below video:\n\n1. When user updates a status of an alert, the `Refetch` only happens on\nthe first `batch`. This behaviour is flaky currently. Even if the user\nis on nth batch, table will fetch 0th batch and reset the user's page\nback to 1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n \n\n\n3. When user clicks `Refresh` manually, then also only first (0th)\n`batch` is fetched, which should have rather fetched all the present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b"}}]}] BACKPORT--> Co-authored-by: Jatin Kathuria <jatin.kathuria@elastic.co>
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…05702) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Fix timeline dynamic batching (#204034)](#204034) - #205893 <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jatin Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-01-07T06:20:30Z","message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","Team:Threat Hunting:Investigations","backport:prev-minor","v8.16.3"],"number":204034,"url":"https://github.com/elastic/kibana/pull/204034","mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204034","number":204034,"mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},{"branch":"8.16","label":"v8.16.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/205674","number":205674,"state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/205676","number":205676,"branch":"8.17","state":"OPEN"}]}] BACKPORT--> --------- Co-authored-by: Philippe Oberti <philippe.oberti@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios (cherry picked from commit 54b4fac) # Conflicts: # x-pack/solutions/security/plugins/security_solution/public/timelines/containers/index.test.tsx # x-pack/solutions/security/plugins/security_solution/public/timelines/containers/index.tsx
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios (cherry picked from commit 54b4fac)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…[ Security Solution ] Fix Refetch logic with new timeline batching (#205893) (#205674) # Backport This will backport the following commits from `main` to `8.16`: - [[Security Solution] Fix timeline dynamic batching (#204034)](#204034) - #205893 <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jatin Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-01-07T06:20:30Z","message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","Team:Threat Hunting:Investigations","backport:prev-minor","v8.16.3"],"number":204034,"url":"https://github.com/elastic/kibana/pull/204034","mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},"sourceBranch":"main","suggestedTargetBranches":["8.16"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204034","number":204034,"mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},{"branch":"8.16","label":"v8.16.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
…[ Security Solution ] Fix Refetch logic with new timeline batching (#205893) (#205676) # Backport This will backport the following commits from `main` to `8.17`: - [[Security Solution] Fix timeline dynamic batching (#204034)](#204034) - #205893 <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jatin Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-01-07T06:20:30Z","message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","Team:Threat Hunting:Investigations","backport:prev-minor","v8.16.3"],"number":204034,"url":"https://github.com/elastic/kibana/pull/204034","mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204034","number":204034,"mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},{"branch":"8.16","label":"v8.16.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/205674","number":205674,"state":"OPEN"}]}] BACKPORT-->
…lastic#205893) ## Summary PR : elastic#204034 fixed some issues with timeline batching. It was not able to fix one of the issue with `Refetch` logic which exists in `main` ( resulting in a flaky test ) and causing some tests to fail in `8.16`, `8.17` and `8.x`. ## Issue Description There are 2 issues with below video: 1. When user updates a status of an alert, the `Refetch` only happens on the first `batch`. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1. https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443 3. When user clicks `Refresh` manually, then also only first (0th) `batch` is fetched, which should have rather fetched all the present batches. https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
…[ Security Solution ] Fix Refetch logic with new timeline batching (#205893) (#210066) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Fix timeline dynamic batching (#204034)](#204034) - [[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)](#205893) <!--- Backport version: 9.6.4 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Jatin Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-01-07T06:20:30Z","message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport missing","v9.0.0","Team:Threat Hunting:Investigations","backport:prev-minor","v8.16.3","v8.19.0"],"title":"[Security Solution] Fix timeline dynamic batching","number":204034,"url":"https://github.com/elastic/kibana/pull/204034","mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204034","number":204034,"mergeCommit":{"message":"[Security Solution] Fix timeline dynamic batching (#204034)\n\n## Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n- https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug where if users fetched multiple batches and then if\r\nuser adds a new column, the value of this new columns will only be\r\nfetched for the latest batch and not old batches.\r\n- This PR fixes that ✅ by cumulatively fetching the data for old batches\r\ntill current batch `iff a new column has been added`.\r\n- For example, if user has already fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched together when a column has been added,\r\notherwise, data will be fetched incrementally.\r\n\r\n### Issue with Elastic search limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we throw error at\r\n10K which should be allowed.\r\n - Error should be thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete code\r\n\r\nBelow files related to old Timeline code are removed as well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},{"branch":"8.16","label":"v8.16.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/205674","number":205674,"state":"OPEN"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"url":"https://github.com/elastic/kibana/pull/205676","number":205676,"branch":"8.17","state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/205702","number":205702,"branch":"8.x","state":"MERGED","mergeCommit":{"sha":"e480de112ae6bc121c4415e2e1726dd551805672","message":"[8.x] [Security Solution] Fix timeline dynamic batching (#204034) (#205702)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.x`:\n- [[Security Solution] Fix timeline dynamic batching\n(#204034)](https://github.com/elastic/kibana/pull/204034)\n - https://github.com/elastic/kibana/pull/205893\n\n<!--- Backport version: 8.9.8 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Jatin\nKathuria\",\"email\":\"jatin.kathuria@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-01-07T06:20:30Z\",\"message\":\"[Security\nSolution] Fix timeline dynamic batching (#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n### Issue with Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n- Timeline had a bug\nwhere if users fetched multiple batches and then if\\r\\nuser adds a new\ncolumn, the value of this new columns will only be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This PR fixes that ✅ by\ncumulatively fetching the data for old batches\\r\\ntill current batch\n`iff a new column has been added`.\\r\\n- For example, if user has already\nfetched the 3rd batch, data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column has been added,\\r\\notherwise, data will be\nfetched incrementally.\\r\\n\\r\\n### Issue with Elastic search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n### Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline code are removed as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"v9.0.0\",\"Team:Threat\nHunting:Investigations\",\"backport:prev-minor\",\"v8.16.3\"],\"number\":204034,\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"mergeCommit\":{\"message\":\"[Security\nSolution] Fix timeline dynamic batching (#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n### Issue with Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n- Timeline had a bug\nwhere if users fetched multiple batches and then if\\r\\nuser adds a new\ncolumn, the value of this new columns will only be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This PR fixes that ✅ by\ncumulatively fetching the data for old batches\\r\\ntill current batch\n`iff a new column has been added`.\\r\\n- For example, if user has already\nfetched the 3rd batch, data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column has been added,\\r\\notherwise, data will be\nfetched incrementally.\\r\\n\\r\\n### Issue with Elastic search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n### Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline code are removed as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"labelRegex\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"number\":204034,\"mergeCommit\":{\"message\":\"[Security\nSolution] Fix timeline dynamic batching (#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n### Issue with Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n- Timeline had a bug\nwhere if users fetched multiple batches and then if\\r\\nuser adds a new\ncolumn, the value of this new columns will only be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This PR fixes that ✅ by\ncumulatively fetching the data for old batches\\r\\ntill current batch\n`iff a new column has been added`.\\r\\n- For example, if user has already\nfetched the 3rd batch, data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column has been added,\\r\\notherwise, data will be\nfetched incrementally.\\r\\n\\r\\n### Issue with Elastic search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n### Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline code are removed as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},{\"branch\":\"8.16\",\"label\":\"v8.16.3\",\"labelRegex\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"url\":\"https://github.com/elastic/kibana/pull/205674\",\"number\":205674,\"state\":\"OPEN\"},{\"url\":\"https://github.com/elastic/kibana/pull/205676\",\"number\":205676,\"branch\":\"8.17\",\"state\":\"OPEN\"}]}]\nBACKPORT-->\n\n---------\n\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>"}}]},{"author":{"name":"Jatin Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-02-05T21:12:38Z","message":"[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)\n\n## Summary\n\nPR : #204034 fixed some issues\nwith timeline batching. It was not able to fix one of the issue with\n`Refetch` logic which exists in `main` ( resulting in a flaky test ) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n## Issue Description\n\nThere are 2 issues with below video:\n\n1. When user updates a status of an alert, the `Refetch` only happens on\nthe first `batch`. This behaviour is flaky currently. Even if the user\nis on nth batch, table will fetch 0th batch and reset the user's page\nback to 1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n \n\n\n3. When user clicks `Refresh` manually, then also only first (0th)\n`batch` is fetched, which should have rather fetched all the present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport","release_note:skip","v9.0.0","Team:Threat Hunting:Investigations","v9.1.0"],"title":"[ Security Solution ] Fix Refetch logic with new timeline batching","number":205893,"url":"https://github.com/elastic/kibana/pull/205893","mergeCommit":{"message":"[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)\n\n## Summary\n\nPR : #204034 fixed some issues\nwith timeline batching. It was not able to fix one of the issue with\n`Refetch` logic which exists in `main` ( resulting in a flaky test ) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n## Issue Description\n\nThere are 2 issues with below video:\n\n1. When user updates a status of an alert, the `Refetch` only happens on\nthe first `batch`. This behaviour is flaky currently. Even if the user\nis on nth batch, table will fetch 0th batch and reset the user's page\nback to 1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n \n\n\n3. When user clicks `Refresh` manually, then also only first (0th)\n`batch` is fetched, which should have rather fetched all the present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/209916","number":209916,"state":"MERGED","mergeCommit":{"sha":"aeaff4957aded708fecfa0e57f16b3cba5f77485","message":"[9.0] [ Security Solution ] Fix Refetch logic with new timeline batching (#205893) (#209916)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.0`:\n- [[ Security Solution ] Fix Refetch logic with new timeline batching\n(#205893)](https://github.com/elastic/kibana/pull/205893)\n\n<!--- Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Jatin\nKathuria\",\"email\":\"jatin.kathuria@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-05T21:12:38Z\",\"message\":\"[\nSecurity Solution ] Fix Refetch logic with new timeline batching\n(#205893)\\n\\n## Summary\\n\\nPR :\nhttps://github.com//pull/204034 fixed some issues\\nwith\ntimeline batching. It was not able to fix one of the issue\nwith\\n`Refetch` logic which exists in `main` ( resulting in a flaky test\n) and\\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\\n\\n##\nIssue Description\\n\\nThere are 2 issues with below video:\\n\\n1. When\nuser updates a status of an alert, the `Refetch` only happens on\\nthe\nfirst `batch`. This behaviour is flaky currently. Even if the user\\nis\non nth batch, table will fetch 0th batch and reset the user's page\\nback\nto\n1.\\n\\n\\n\\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\\n\n\\n\\n\\n3. When user clicks `Refresh` manually, then also only first\n(0th)\\n`batch` is fetched, which should have rather fetched all the\npresent\\nbatches.\\n\\n\\n\\n\\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\\n\\n\\n\\n###\nChecklist\\n\\nCheck the PR satisfies following conditions. \\n\\nReviewers\nshould verify this PR satisfies this list as well.\\n\\n\\n- [x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common\nscenarios\",\"sha\":\"54b4fac705c231b52396d70906f3259f9b129a3b\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"backport\",\"release_note:skip\",\"Team:Threat\nHunting:Investigations\",\"v8.9.1\",\"v9.1.0\"],\"title\":\"[ Security Solution\n] Fix Refetch logic with new timeline\nbatching\",\"number\":205893,\"url\":\"https://github.com/elastic/kibana/pull/205893\",\"mergeCommit\":{\"message\":\"[\nSecurity Solution ] Fix Refetch logic with new timeline batching\n(#205893)\\n\\n## Summary\\n\\nPR :\nhttps://github.com//pull/204034 fixed some issues\\nwith\ntimeline batching. It was not able to fix one of the issue\nwith\\n`Refetch` logic which exists in `main` ( resulting in a flaky test\n) and\\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\\n\\n##\nIssue Description\\n\\nThere are 2 issues with below video:\\n\\n1. When\nuser updates a status of an alert, the `Refetch` only happens on\\nthe\nfirst `batch`. This behaviour is flaky currently. Even if the user\\nis\non nth batch, table will fetch 0th batch and reset the user's page\\nback\nto\n1.\\n\\n\\n\\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\\n\n\\n\\n\\n3. When user clicks `Refresh` manually, then also only first\n(0th)\\n`batch` is fetched, which should have rather fetched all the\npresent\\nbatches.\\n\\n\\n\\n\\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\\n\\n\\n\\n###\nChecklist\\n\\nCheck the PR satisfies following conditions. \\n\\nReviewers\nshould verify this PR satisfies this list as well.\\n\\n\\n- [x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common\nscenarios\",\"sha\":\"54b4fac705c231b52396d70906f3259f9b129a3b\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.9\"],\"targetPullRequestStates\":[{\"branch\":\"8.9\",\"label\":\"v8.9.1\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/205893\",\"number\":205893,\"mergeCommit\":{\"message\":\"[\nSecurity Solution ] Fix Refetch logic with new timeline batching\n(#205893)\\n\\n## Summary\\n\\nPR :\nhttps://github.com//pull/204034 fixed some issues\\nwith\ntimeline batching. It was not able to fix one of the issue\nwith\\n`Refetch` logic which exists in `main` ( resulting in a flaky test\n) and\\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\\n\\n##\nIssue Description\\n\\nThere are 2 issues with below video:\\n\\n1. When\nuser updates a status of an alert, the `Refetch` only happens on\\nthe\nfirst `batch`. This behaviour is flaky currently. Even if the user\\nis\non nth batch, table will fetch 0th batch and reset the user's page\\nback\nto\n1.\\n\\n\\n\\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\\n\n\\n\\n\\n3. When user clicks `Refresh` manually, then also only first\n(0th)\\n`batch` is fetched, which should have rather fetched all the\npresent\\nbatches.\\n\\n\\n\\n\\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\\n\\n\\n\\n###\nChecklist\\n\\nCheck the PR satisfies following conditions. \\n\\nReviewers\nshould verify this PR satisfies this list as well.\\n\\n\\n- [x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common\nscenarios\",\"sha\":\"54b4fac705c231b52396d70906f3259f9b129a3b\"}}]}]\nBACKPORT-->\n\nCo-authored-by: Jatin Kathuria <jatin.kathuria@elastic.co>"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205893","number":205893,"mergeCommit":{"message":"[ Security Solution ] Fix Refetch logic with new timeline batching (#205893)\n\n## Summary\n\nPR : #204034 fixed some issues\nwith timeline batching. It was not able to fix one of the issue with\n`Refetch` logic which exists in `main` ( resulting in a flaky test ) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n## Issue Description\n\nThere are 2 issues with below video:\n\n1. When user updates a status of an alert, the `Refetch` only happens on\nthe first `batch`. This behaviour is flaky currently. Even if the user\nis on nth batch, table will fetch 0th batch and reset the user's page\nback to 1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n \n\n\n3. When user clicks `Refresh` manually, then also only first (0th)\n`batch` is fetched, which should have rather fetched all the present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b"}},{"url":"https://github.com/elastic/kibana/pull/205676","number":205676,"branch":"8.17","state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/205702","number":205702,"branch":"8.x","state":"MERGED","mergeCommit":{"sha":"e480de112ae6bc121c4415e2e1726dd551805672","message":"[8.x] [Security Solution] Fix timeline dynamic batching (#204034) (#205702)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.x`:\n- [[Security Solution] Fix timeline dynamic batching\n(#204034)](https://github.com/elastic/kibana/pull/204034)\n - https://github.com/elastic/kibana/pull/205893\n\n<!--- Backport version: 8.9.8 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Jatin\nKathuria\",\"email\":\"jatin.kathuria@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-01-07T06:20:30Z\",\"message\":\"[Security\nSolution] Fix timeline dynamic batching (#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n### Issue with Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n- Timeline had a bug\nwhere if users fetched multiple batches and then if\\r\\nuser adds a new\ncolumn, the value of this new columns will only be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This PR fixes that ✅ by\ncumulatively fetching the data for old batches\\r\\ntill current batch\n`iff a new column has been added`.\\r\\n- For example, if user has already\nfetched the 3rd batch, data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column has been added,\\r\\notherwise, data will be\nfetched incrementally.\\r\\n\\r\\n### Issue with Elastic search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n### Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline code are removed as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"v9.0.0\",\"Team:Threat\nHunting:Investigations\",\"backport:prev-minor\",\"v8.16.3\"],\"number\":204034,\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"mergeCommit\":{\"message\":\"[Security\nSolution] Fix timeline dynamic batching (#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n### Issue with Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n- Timeline had a bug\nwhere if users fetched multiple batches and then if\\r\\nuser adds a new\ncolumn, the value of this new columns will only be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This PR fixes that ✅ by\ncumulatively fetching the data for old batches\\r\\ntill current batch\n`iff a new column has been added`.\\r\\n- For example, if user has already\nfetched the 3rd batch, data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column has been added,\\r\\notherwise, data will be\nfetched incrementally.\\r\\n\\r\\n### Issue with Elastic search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n### Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline code are removed as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"labelRegex\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"number\":204034,\"mergeCommit\":{\"message\":\"[Security\nSolution] Fix timeline dynamic batching (#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n### Issue with Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n- Timeline had a bug\nwhere if users fetched multiple batches and then if\\r\\nuser adds a new\ncolumn, the value of this new columns will only be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This PR fixes that ✅ by\ncumulatively fetching the data for old batches\\r\\ntill current batch\n`iff a new column has been added`.\\r\\n- For example, if user has already\nfetched the 3rd batch, data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column has been added,\\r\\notherwise, data will be\nfetched incrementally.\\r\\n\\r\\n### Issue with Elastic search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n### Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline code are removed as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},{\"branch\":\"8.16\",\"label\":\"v8.16.3\",\"labelRegex\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"url\":\"https://github.com/elastic/kibana/pull/205674\",\"number\":205674,\"state\":\"OPEN\"},{\"url\":\"https://github.com/elastic/kibana/pull/205676\",\"number\":205676,\"branch\":\"8.17\",\"state\":\"OPEN\"}]}]\nBACKPORT-->\n\n---------\n\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>"}},{"url":"https://github.com/elastic/kibana/pull/205674","number":205674,"branch":"8.16","state":"OPEN"}]}] BACKPORT-->
Summary
PR : #204034 fixed some issues with timeline batching. It was not able to fix one of the issue with
Refetchlogic which exists inmain( resulting in a flaky test ) and causing some tests to fail in8.16,8.17and8.x.Issue Description
There are 2 issues with below video:
Refetchonly happens on the firstbatch. This behaviour is flaky currently. Even if the user is on nth batch, table will fetch 0th batch and reset the user's page back to 1.refetch_update_status.mp4
Refreshmanually, then also only first (0th)batchis fetched, which should have rather fetched all the present batches.reftech_refresh_button.mov
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.