Description

Pagination for agent is broken after 10k (or whatever the index.max_result_window setting is for the cluster) agents. Elasticsearch offers various ways to paginate search results, can we use one of them to improve this experience?: https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html

Tasks

• Improve UX for >10k agents; ideally there would be no error, and UI can display all agents beyond max result setting
  • This should be done using the point in time API provided by Elasticsearch. (note: below there was a discussion using saved objects' support for this, but these objects are no longer stored as saved objects so it's no longer relevant)
• Remove "First 10,000" limitation on UI
• Investigate if there is a similar limitation on backend when performing bulk actions (Jen doesn't think so, as we offer a "select all agents beyond 10k" option to perform actions against, but double check just in case)

How to reproduce

enroll 10k agents try to navigate to the last page of the agent listing

the details of the error

Result window is too large, from + size must be less than or equal to: [10000] but was [10260]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.