Add KQL functionality in the find function of the saved objects#41136
Add KQL functionality in the find function of the saved objects#41136XavierM merged 14 commits intoelastic:masterfrom
Conversation
|
Mike Cote brought up another use case that could be solved by having KQL: https://github.com/elastic/kibana/pull/39829/files#diff-5598f367eaa6aef5c6bbe4654d764a9bR130 |
d6e65c6 to
3a51743
Compare
|
Pinging @elastic/kibana-platform |
rudolf
left a comment
There was a problem hiding this comment.
Left some initial comments
src/core/server/saved_objects/service/lib/search_dsl/query_params.ts
Outdated
Show resolved
Hide resolved
3a51743 to
38bae52
Compare
|
Edit: added a 3rd option. @elastic/kibana-platform In order to evaluate KQL expressions the KQL parser requires a There are two ways to collect this information:
If we use (1) it means we'll have to move this service to Core, I haven't seen this discussed, but I assume the current plan is for it to live in the data plugin. (2) is a bit more complex and it means maintaining a field type -> {searchable, aggregatable} mapping instead of leveraging the ES API which is guaranteed to be up to date. The way @XavierM implemented it now the SavedObjects Repository only requires the compiled static index patterns, so it's easy to swap around how we construct these when we move everything to Core. |
172d56a to
c8af3d0
Compare
rudolf
left a comment
There was a problem hiding this comment.
Github stopped letting me add comments 🚫 so submitting what I have.
src/legacy/server/saved_objects/lib/create_saved_objects_stream_from_ndjson.ts
Outdated
Show resolved
Hide resolved
05305c6 to
ccb5da5
Compare
This comment has been minimized.
This comment has been minimized.
ccb5da5 to
78bf464
Compare
6f9ef53 to
9ac6de7
Compare
💚 Build Succeeded |
9ac6de7 to
e72f7f5
Compare
This comment has been minimized.
This comment has been minimized.
wip rename variable from KQL to filter, fix unit test + add new ones miss security pluggins review I fix api changes refactor after reviewing with Rudolf fix type review III review IV for security put back allowed logic back to return empty results remove StaticIndexPattern review V fix core_api_changes fix type
💚 Build Succeeded |
💔 Build Failed |
Summary
The SIEM app is migrating to using SavedObjects which also means migrating all their queries. Since these are written in KQL rewriting them as Simple Query Strings will be time-consuming and error-prone. In addition, Simple Query String doesn't support range queries.
Since KQL is designed as a simple filter language it doesn't replace searching with Simple Query Strings in Saved Objects but rather can be used in addition to constructing more powerful queries.
Dev Docs
SavedObjectsClient.find now supports filtering using a KQL string, with the caveat that if you filter with an attribute from your type saved object, it should look like this:
savedObjectType.attributes.name: "SayMyName". However, If you use a direct attribute of a saved object likeupdatedAt, you will have to define your filter like this:savedObjectType.updatedAt > 2018-12-22.Checklist
Use
strikethroughsto remove checklist items you don't feel are applicable to this PR.[ ] This was checked for cross-browser compatibility, including a check against IE11[ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support[ ] This was checked for keyboard-only and screenreader accessibilityFor maintainers
[ ] This was checked for breaking API changes and was labeled appropriately[ ] This includes a feature addition or change that requires a release note and was labeled appropriately