Skip to content

[8.x] [Security Solution] Fix timeline dynamic batching (#204034)#205702

Merged
logeekal merged 7 commits intoelastic:8.xfrom
logeekal:backport/8.x/pr-204034
Feb 6, 2025
Merged

[8.x] [Security Solution] Fix timeline dynamic batching (#204034)#205702
logeekal merged 7 commits intoelastic:8.xfrom
logeekal:backport/8.x/pr-204034

Conversation

@logeekal
Copy link
Copy Markdown
Contributor

@logeekal logeekal commented Jan 7, 2025

logeekal and others added 2 commits January 7, 2025 11:29
## Summary

Handles :

### Issue with Batches
- elastic#201405
- Timeline had a bug where if users fetched multiple batches and then if
user adds a new column, the value of this new columns will only be
fetched for the latest batch and not old batches.
- This PR fixes that ✅ by cumulatively fetching the data for old batches
till current batch `iff a new column has been added`.
- For example, if user has already fetched the 3rd batch, data for
1st,2nd and 3rd will be fetched together when a column has been added,
otherwise, data will be fetched incrementally.

### Issue with Elastic search limit

- Elastic search has a limit of 10K hits at max but we throw error at
10K which should be allowed.
    - Error should be thrown at anything `>10K`. 10001 for example.
    - ✅  This PR fixes that just for timeline by allowing 10K hits.

### Removal of obsolete code

Below files related to old Timeline code are removed as well:
-
x-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx
-
x-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx

---------

Co-authored-by: Philippe Oberti <philippe.oberti@elastic.co>
@logeekal logeekal added the backport This PR is a backport of another PR label Jan 7, 2025
@logeekal logeekal enabled auto-merge (squash) January 7, 2025 10:52
logeekal and others added 4 commits February 6, 2025 09:59
…lastic#205893)

## Summary

PR : elastic#204034 fixed some issues
with timeline batching. It was not able to fix one of the issue with
`Refetch` logic which exists in `main` ( resulting in a flaky test ) and
causing some tests to fail in `8.16`, `8.17` and `8.x`.

## Issue Description

There are 2 issues with below video:

1. When user updates a status of an alert, the `Refetch` only happens on
the first `batch`. This behaviour is flaky currently. Even if the user
is on nth batch, table will fetch 0th batch and reset the user's page
back to 1.



https://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443
     


3. When user clicks `Refresh` manually, then also only first (0th)
`batch` is fetched, which should have rather fetched all the present
batches.




https://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0



### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.


- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
@logeekal logeekal merged commit e480de1 into elastic:8.x Feb 6, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 18.8MB 18.8MB +1.6KB

History

logeekal added a commit that referenced this pull request Feb 8, 2025
…[ Security Solution ] Fix Refetch logic with new timeline batching (#205893) (#210066)

# Backport

This will backport the following commits from `main` to `8.18`:
- [[Security Solution] Fix timeline dynamic batching
(#204034)](#204034)
- [[ Security Solution ] Fix Refetch logic with new timeline batching
(#205893)](#205893)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Jatin
Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-01-07T06:20:30Z","message":"[Security
Solution] Fix timeline dynamic batching (#204034)\n\n##
Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n-
https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug
where if users fetched multiple batches and then if\r\nuser adds a new
column, the value of this new columns will only be\r\nfetched for the
latest batch and not old batches.\r\n- This PR fixes that ✅ by
cumulatively fetching the data for old batches\r\ntill current batch
`iff a new column has been added`.\r\n- For example, if user has already
fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched
together when a column has been added,\r\notherwise, data will be
fetched incrementally.\r\n\r\n### Issue with Elastic search
limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we
throw error at\r\n10K which should be allowed.\r\n - Error should be
thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that
just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete
code\r\n\r\nBelow files related to old Timeline code are removed as
well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by:
Philippe Oberti
<philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport
missing","v9.0.0","Team:Threat
Hunting:Investigations","backport:prev-minor","v8.16.3","v8.19.0"],"title":"[Security
Solution] Fix timeline dynamic
batching","number":204034,"url":"https://github.com/elastic/kibana/pull/204034","mergeCommit":{"message":"[Security
Solution] Fix timeline dynamic batching (#204034)\n\n##
Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n-
https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug
where if users fetched multiple batches and then if\r\nuser adds a new
column, the value of this new columns will only be\r\nfetched for the
latest batch and not old batches.\r\n- This PR fixes that ✅ by
cumulatively fetching the data for old batches\r\ntill current batch
`iff a new column has been added`.\r\n- For example, if user has already
fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched
together when a column has been added,\r\notherwise, data will be
fetched incrementally.\r\n\r\n### Issue with Elastic search
limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we
throw error at\r\n10K which should be allowed.\r\n - Error should be
thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that
just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete
code\r\n\r\nBelow files related to old Timeline code are removed as
well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by:
Philippe Oberti
<philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204034","number":204034,"mergeCommit":{"message":"[Security
Solution] Fix timeline dynamic batching (#204034)\n\n##
Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n-
https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug
where if users fetched multiple batches and then if\r\nuser adds a new
column, the value of this new columns will only be\r\nfetched for the
latest batch and not old batches.\r\n- This PR fixes that ✅ by
cumulatively fetching the data for old batches\r\ntill current batch
`iff a new column has been added`.\r\n- For example, if user has already
fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched
together when a column has been added,\r\notherwise, data will be
fetched incrementally.\r\n\r\n### Issue with Elastic search
limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we
throw error at\r\n10K which should be allowed.\r\n - Error should be
thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that
just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete
code\r\n\r\nBelow files related to old Timeline code are removed as
well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by:
Philippe Oberti
<philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},{"branch":"8.16","label":"v8.16.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/205674","number":205674,"state":"OPEN"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"url":"https://github.com/elastic/kibana/pull/205676","number":205676,"branch":"8.17","state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/205702","number":205702,"branch":"8.x","state":"MERGED","mergeCommit":{"sha":"e480de112ae6bc121c4415e2e1726dd551805672","message":"[8.x]
[Security Solution] Fix timeline dynamic batching (#204034)
(#205702)\n\n# Backport\n\nThis will backport the following commits from
`main` to `8.x`:\n- [[Security Solution] Fix timeline dynamic
batching\n(#204034)](https://github.com/elastic/kibana/pull/204034)\n -
https://github.com/elastic/kibana/pull/205893\n\n<!--- Backport version:
8.9.8 -->\n\n### Questions ?\nPlease refer to the [Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Jatin\nKathuria\",\"email\":\"jatin.kathuria@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-01-07T06:20:30Z\",\"message\":\"[Security\nSolution]
Fix timeline dynamic batching
(#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n###
Issue with
Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n-
Timeline had a bug\nwhere if users fetched multiple batches and then
if\\r\\nuser adds a new\ncolumn, the value of this new columns will only
be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This
PR fixes that ✅ by\ncumulatively fetching the data for old
batches\\r\\ntill current batch\n`iff a new column has been
added`.\\r\\n- For example, if user has already\nfetched the 3rd batch,
data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column
has been added,\\r\\notherwise, data will be\nfetched
incrementally.\\r\\n\\r\\n### Issue with Elastic
search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max
but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error
should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This
PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n###
Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline
code are removed
as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe
Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"v9.0.0\",\"Team:Threat\nHunting:Investigations\",\"backport:prev-minor\",\"v8.16.3\"],\"number\":204034,\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"mergeCommit\":{\"message\":\"[Security\nSolution]
Fix timeline dynamic batching
(#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n###
Issue with
Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n-
Timeline had a bug\nwhere if users fetched multiple batches and then
if\\r\\nuser adds a new\ncolumn, the value of this new columns will only
be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This
PR fixes that ✅ by\ncumulatively fetching the data for old
batches\\r\\ntill current batch\n`iff a new column has been
added`.\\r\\n- For example, if user has already\nfetched the 3rd batch,
data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column
has been added,\\r\\notherwise, data will be\nfetched
incrementally.\\r\\n\\r\\n### Issue with Elastic
search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max
but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error
should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This
PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n###
Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline
code are removed
as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe
Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"labelRegex\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"number\":204034,\"mergeCommit\":{\"message\":\"[Security\nSolution]
Fix timeline dynamic batching
(#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n###
Issue with
Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n-
Timeline had a bug\nwhere if users fetched multiple batches and then
if\\r\\nuser adds a new\ncolumn, the value of this new columns will only
be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This
PR fixes that ✅ by\ncumulatively fetching the data for old
batches\\r\\ntill current batch\n`iff a new column has been
added`.\\r\\n- For example, if user has already\nfetched the 3rd batch,
data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column
has been added,\\r\\notherwise, data will be\nfetched
incrementally.\\r\\n\\r\\n### Issue with Elastic
search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max
but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error
should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This
PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n###
Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline
code are removed
as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe
Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},{\"branch\":\"8.16\",\"label\":\"v8.16.3\",\"labelRegex\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"url\":\"https://github.com/elastic/kibana/pull/205674\",\"number\":205674,\"state\":\"OPEN\"},{\"url\":\"https://github.com/elastic/kibana/pull/205676\",\"number\":205676,\"branch\":\"8.17\",\"state\":\"OPEN\"}]}]\nBACKPORT-->\n\n---------\n\nCo-authored-by:
Philippe Oberti <philippe.oberti@elastic.co>\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>"}}]},{"author":{"name":"Jatin
Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-02-05T21:12:38Z","message":"[
Security Solution ] Fix Refetch logic with new timeline batching
(#205893)\n\n## Summary\n\nPR :
#204034 fixed some issues\nwith
timeline batching. It was not able to fix one of the issue
with\n`Refetch` logic which exists in `main` ( resulting in a flaky test
) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n##
Issue Description\n\nThere are 2 issues with below video:\n\n1. When
user updates a status of an alert, the `Refetch` only happens on\nthe
first `batch`. This behaviour is flaky currently. Even if the user\nis
on nth batch, table will fetch 0th batch and reset the user's page\nback
to
1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n
\n\n\n3. When user clicks `Refresh` manually, then also only first
(0th)\n`batch` is fetched, which should have rather fetched all the
present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport","release_note:skip","v9.0.0","Team:Threat
Hunting:Investigations","v9.1.0"],"title":"[ Security Solution ] Fix
Refetch logic with new timeline
batching","number":205893,"url":"https://github.com/elastic/kibana/pull/205893","mergeCommit":{"message":"[
Security Solution ] Fix Refetch logic with new timeline batching
(#205893)\n\n## Summary\n\nPR :
#204034 fixed some issues\nwith
timeline batching. It was not able to fix one of the issue
with\n`Refetch` logic which exists in `main` ( resulting in a flaky test
) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n##
Issue Description\n\nThere are 2 issues with below video:\n\n1. When
user updates a status of an alert, the `Refetch` only happens on\nthe
first `batch`. This behaviour is flaky currently. Even if the user\nis
on nth batch, table will fetch 0th batch and reset the user's page\nback
to
1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n
\n\n\n3. When user clicks `Refresh` manually, then also only first
(0th)\n`batch` is fetched, which should have rather fetched all the
present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/209916","number":209916,"state":"MERGED","mergeCommit":{"sha":"aeaff4957aded708fecfa0e57f16b3cba5f77485","message":"[9.0]
[ Security Solution ] Fix Refetch logic with new timeline batching
(#205893) (#209916)\n\n# Backport\n\nThis will backport the following
commits from `main` to `9.0`:\n- [[ Security Solution ] Fix Refetch
logic with new timeline
batching\n(#205893)](https://github.com/elastic/kibana/pull/205893)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Jatin\nKathuria\",\"email\":\"jatin.kathuria@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-05T21:12:38Z\",\"message\":\"[\nSecurity
Solution ] Fix Refetch logic with new timeline
batching\n(#205893)\\n\\n## Summary\\n\\nPR
:\nhttps://github.com//pull/204034 fixed some
issues\\nwith\ntimeline batching. It was not able to fix one of the
issue\nwith\\n`Refetch` logic which exists in `main` ( resulting in a
flaky test\n) and\\ncausing some tests to fail in `8.16`, `8.17` and
`8.x`.\\n\\n##\nIssue Description\\n\\nThere are 2 issues with below
video:\\n\\n1. When\nuser updates a status of an alert, the `Refetch`
only happens on\\nthe\nfirst `batch`. This behaviour is flaky currently.
Even if the user\\nis\non nth batch, table will fetch 0th batch and
reset the user's
page\\nback\nto\n1.\\n\\n\\n\\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\\n\n\\n\\n\\n3.
When user clicks `Refresh` manually, then also only
first\n(0th)\\n`batch` is fetched, which should have rather fetched all
the\npresent\\nbatches.\\n\\n\\n\\n\\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\\n\\n\\n\\n###\nChecklist\\n\\nCheck
the PR satisfies following conditions. \\n\\nReviewers\nshould verify
this PR satisfies this list as well.\\n\\n\\n- [x] [Unit
or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated
or added to match the most
common\nscenarios\",\"sha\":\"54b4fac705c231b52396d70906f3259f9b129a3b\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"backport\",\"release_note:skip\",\"Team:Threat\nHunting:Investigations\",\"v8.9.1\",\"v9.1.0\"],\"title\":\"[
Security Solution\n] Fix Refetch logic with new
timeline\nbatching\",\"number\":205893,\"url\":\"https://github.com/elastic/kibana/pull/205893\",\"mergeCommit\":{\"message\":\"[\nSecurity
Solution ] Fix Refetch logic with new timeline
batching\n(#205893)\\n\\n## Summary\\n\\nPR
:\nhttps://github.com//pull/204034 fixed some
issues\\nwith\ntimeline batching. It was not able to fix one of the
issue\nwith\\n`Refetch` logic which exists in `main` ( resulting in a
flaky test\n) and\\ncausing some tests to fail in `8.16`, `8.17` and
`8.x`.\\n\\n##\nIssue Description\\n\\nThere are 2 issues with below
video:\\n\\n1. When\nuser updates a status of an alert, the `Refetch`
only happens on\\nthe\nfirst `batch`. This behaviour is flaky currently.
Even if the user\\nis\non nth batch, table will fetch 0th batch and
reset the user's
page\\nback\nto\n1.\\n\\n\\n\\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\\n\n\\n\\n\\n3.
When user clicks `Refresh` manually, then also only
first\n(0th)\\n`batch` is fetched, which should have rather fetched all
the\npresent\\nbatches.\\n\\n\\n\\n\\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\\n\\n\\n\\n###\nChecklist\\n\\nCheck
the PR satisfies following conditions. \\n\\nReviewers\nshould verify
this PR satisfies this list as well.\\n\\n\\n- [x] [Unit
or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated
or added to match the most
common\nscenarios\",\"sha\":\"54b4fac705c231b52396d70906f3259f9b129a3b\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.9\"],\"targetPullRequestStates\":[{\"branch\":\"8.9\",\"label\":\"v8.9.1\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/205893\",\"number\":205893,\"mergeCommit\":{\"message\":\"[\nSecurity
Solution ] Fix Refetch logic with new timeline
batching\n(#205893)\\n\\n## Summary\\n\\nPR
:\nhttps://github.com//pull/204034 fixed some
issues\\nwith\ntimeline batching. It was not able to fix one of the
issue\nwith\\n`Refetch` logic which exists in `main` ( resulting in a
flaky test\n) and\\ncausing some tests to fail in `8.16`, `8.17` and
`8.x`.\\n\\n##\nIssue Description\\n\\nThere are 2 issues with below
video:\\n\\n1. When\nuser updates a status of an alert, the `Refetch`
only happens on\\nthe\nfirst `batch`. This behaviour is flaky currently.
Even if the user\\nis\non nth batch, table will fetch 0th batch and
reset the user's
page\\nback\nto\n1.\\n\\n\\n\\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\\n\n\\n\\n\\n3.
When user clicks `Refresh` manually, then also only
first\n(0th)\\n`batch` is fetched, which should have rather fetched all
the\npresent\\nbatches.\\n\\n\\n\\n\\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\\n\\n\\n\\n###\nChecklist\\n\\nCheck
the PR satisfies following conditions. \\n\\nReviewers\nshould verify
this PR satisfies this list as well.\\n\\n\\n- [x] [Unit
or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated
or added to match the most
common\nscenarios\",\"sha\":\"54b4fac705c231b52396d70906f3259f9b129a3b\"}}]}]\nBACKPORT-->\n\nCo-authored-by:
Jatin Kathuria
<jatin.kathuria@elastic.co>"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205893","number":205893,"mergeCommit":{"message":"[
Security Solution ] Fix Refetch logic with new timeline batching
(#205893)\n\n## Summary\n\nPR :
#204034 fixed some issues\nwith
timeline batching. It was not able to fix one of the issue
with\n`Refetch` logic which exists in `main` ( resulting in a flaky test
) and\ncausing some tests to fail in `8.16`, `8.17` and `8.x`.\n\n##
Issue Description\n\nThere are 2 issues with below video:\n\n1. When
user updates a status of an alert, the `Refetch` only happens on\nthe
first `batch`. This behaviour is flaky currently. Even if the user\nis
on nth batch, table will fetch 0th batch and reset the user's page\nback
to
1.\n\n\n\nhttps://github.com/user-attachments/assets/eaf88a82-0e9b-4743-8b2d-60fd327a2443\n
\n\n\n3. When user clicks `Refresh` manually, then also only first
(0th)\n`batch` is fetched, which should have rather fetched all the
present\nbatches.\n\n\n\n\nhttps://github.com/user-attachments/assets/8d578ce3-4f24-4e70-bc3a-ed6ba99167a0\n\n\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"54b4fac705c231b52396d70906f3259f9b129a3b"}},{"url":"https://github.com/elastic/kibana/pull/205676","number":205676,"branch":"8.17","state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/205702","number":205702,"branch":"8.x","state":"MERGED","mergeCommit":{"sha":"e480de112ae6bc121c4415e2e1726dd551805672","message":"[8.x]
[Security Solution] Fix timeline dynamic batching (#204034)
(#205702)\n\n# Backport\n\nThis will backport the following commits from
`main` to `8.x`:\n- [[Security Solution] Fix timeline dynamic
batching\n(#204034)](https://github.com/elastic/kibana/pull/204034)\n -
https://github.com/elastic/kibana/pull/205893\n\n<!--- Backport version:
8.9.8 -->\n\n### Questions ?\nPlease refer to the [Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Jatin\nKathuria\",\"email\":\"jatin.kathuria@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-01-07T06:20:30Z\",\"message\":\"[Security\nSolution]
Fix timeline dynamic batching
(#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n###
Issue with
Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n-
Timeline had a bug\nwhere if users fetched multiple batches and then
if\\r\\nuser adds a new\ncolumn, the value of this new columns will only
be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This
PR fixes that ✅ by\ncumulatively fetching the data for old
batches\\r\\ntill current batch\n`iff a new column has been
added`.\\r\\n- For example, if user has already\nfetched the 3rd batch,
data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column
has been added,\\r\\notherwise, data will be\nfetched
incrementally.\\r\\n\\r\\n### Issue with Elastic
search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max
but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error
should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This
PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n###
Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline
code are removed
as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe
Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"v9.0.0\",\"Team:Threat\nHunting:Investigations\",\"backport:prev-minor\",\"v8.16.3\"],\"number\":204034,\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"mergeCommit\":{\"message\":\"[Security\nSolution]
Fix timeline dynamic batching
(#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n###
Issue with
Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n-
Timeline had a bug\nwhere if users fetched multiple batches and then
if\\r\\nuser adds a new\ncolumn, the value of this new columns will only
be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This
PR fixes that ✅ by\ncumulatively fetching the data for old
batches\\r\\ntill current batch\n`iff a new column has been
added`.\\r\\n- For example, if user has already\nfetched the 3rd batch,
data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column
has been added,\\r\\notherwise, data will be\nfetched
incrementally.\\r\\n\\r\\n### Issue with Elastic
search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max
but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error
should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This
PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n###
Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline
code are removed
as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe
Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"labelRegex\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/204034\",\"number\":204034,\"mergeCommit\":{\"message\":\"[Security\nSolution]
Fix timeline dynamic batching
(#204034)\\n\\n##\nSummary\\r\\n\\r\\nHandles :\\r\\n\\r\\n\\r\\n###
Issue with
Batches\\r\\n-\nhttps://github.com//issues/201405\\r\\n-
Timeline had a bug\nwhere if users fetched multiple batches and then
if\\r\\nuser adds a new\ncolumn, the value of this new columns will only
be\\r\\nfetched for the\nlatest batch and not old batches.\\r\\n- This
PR fixes that ✅ by\ncumulatively fetching the data for old
batches\\r\\ntill current batch\n`iff a new column has been
added`.\\r\\n- For example, if user has already\nfetched the 3rd batch,
data for\\r\\n1st,2nd and 3rd will be fetched\ntogether when a column
has been added,\\r\\notherwise, data will be\nfetched
incrementally.\\r\\n\\r\\n### Issue with Elastic
search\nlimit\\r\\n\\r\\n- Elastic search has a limit of 10K hits at max
but we\nthrow error at\\r\\n10K which should be allowed.\\r\\n - Error
should be\nthrown at anything `>10K`. 10001 for example.\\r\\n - ✅ This
PR fixes that\njust for timeline by allowing 10K hits.\\r\\n\\r\\n###
Removal of obsolete\ncode\\r\\n\\r\\nBelow files related to old Timeline
code are removed
as\nwell:\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\\r\\n-\\r\\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:\nPhilippe
Oberti\n<philippe.oberti@elastic.co>\",\"sha\":\"088169f446788f9fa8800d77817881524514943e\"}},{\"branch\":\"8.16\",\"label\":\"v8.16.3\",\"labelRegex\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"url\":\"https://github.com/elastic/kibana/pull/205674\",\"number\":205674,\"state\":\"OPEN\"},{\"url\":\"https://github.com/elastic/kibana/pull/205676\",\"number\":205676,\"branch\":\"8.17\",\"state\":\"OPEN\"}]}]\nBACKPORT-->\n\n---------\n\nCo-authored-by:
Philippe Oberti <philippe.oberti@elastic.co>\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>"}},{"url":"https://github.com/elastic/kibana/pull/205674","number":205674,"branch":"8.16","state":"OPEN"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants