[data views] Provide method of excluding data tiers when getting field list#167946
[data views] Provide method of excluding data tiers when getting field list#167946mattkime merged 15 commits intoelastic:mainfrom
Conversation
|
/ci |
| export const dataTiersUiSettingsConfig: Record<string, UiSettingsParams> = { | ||
| [DATA_VIEWS_FIELDS_EXCLUDED_TIERS]: { | ||
| name: i18n.translate('dataViews.advancedSettings.dataTiersName', { | ||
| defaultMessage: 'Data tiers excluded from field requests', | ||
| }), | ||
| value: '', | ||
| type: 'string', | ||
| description: i18n.translate('dataViews.advancedSettings.dataTiersText', { | ||
| defaultMessage: | ||
| 'Exclude fields from specified tiers (such as data_frozen) for faster performance.', | ||
| }), | ||
| schema: schema.string(), | ||
| }, | ||
| }; | ||
|
|
|
@mattkime qq: apart from tests, this is done, right? |
|
@kertal Exactly! |
|
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery) |
|
great! 👍 |
There was a problem hiding this comment.
Code-only review. The code changes look good to me, just one minor piece of feedback so far. Including functional tests once we've figured out how would be great, although I'm also not opposed to merging with just unit tests and the tests @kertal suggested for now with additional functional tests as a followup if we think it could be a bit before we're able to add them.
davismcphee
left a comment
There was a problem hiding this comment.
Tested locally using the following sample data (thanks @kertal), and confirmed that adding data_cold to the exclusion list properly excludes cold_field from both the fields and fields_for_wildcard results:
PUT /test-cold
{
"settings": {
"index.routing.allocation.include._tier_preference": "data_cold"
}
}
PUT /test-cold/_doc/1
{
"cold_field": "test"
}
PUT /test-normal/_doc/1
{
"normal_field": "test"
}
LGTM 👍 Would you mind adding a followup issue to add functional tests once the necessary QA functionality is merged?
| value: '', | ||
| type: 'string', | ||
| description: i18n.translate('dataViews.advancedSettings.dataTiersText', { | ||
| defaultMessage: |
There was a problem hiding this comment.
Is it worth indicating the expected comma-separated format, or do we think users will know what to expect for this?
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: cc @mattkime |
kertal
left a comment
There was a problem hiding this comment.
Tested a-la-carte, excluding data_cold, work as expected, fields are showing up unmapped 👍 thx for bringing this over the finishing line in no time
|
Is this something we are also considering for Unified Search when users fetch fields for filtering, queriying, etc.? |
|
@teresaalvarezsoler yes, since AFAIK these fields are provided by data views, so this also applies to these fields (there might be exceptions) |
* main: use build hash in FTR tests [Security Solution] Fix moderate typo (elastic#175883) [Fleet] Fix conflicting dynamic template mappings for intermediate objects (elastic#175970) [Visualize] Prevent overwriting managed content (elastic#175274) [SLO] Add/edit form mark optional fields (elastic#175807) skip failing test suite (elastic#175984) [data views] Provide method of excluding data tiers when getting field list (elastic#167946) [Dataset quality] State management (elastic#174906) [Cloud Security] add is_internal config option for outputs (elastic#175546)
…d list (elastic#167946) ## Summary This PR implements an advanced setting that allows the exclusion of listed data tiers when getting a field list. The expected common use case would be excluding frozen indices to speed up slow field caps calls. There is no serverless functionality since serverless doesn't have data tiers.
…d list (elastic#167946) ## Summary This PR implements an advanced setting that allows the exclusion of listed data tiers when getting a field list. The expected common use case would be excluding frozen indices to speed up slow field caps calls. There is no serverless functionality since serverless doesn't have data tiers.
Summary
This PR implements an Advanced setting (data_views:fields_excluded_data_tiers) that allows the exclusion of listed data tiers when getting a field list. The expected common use case would be excluding frozen indices to speed up slow field caps calls.
There is no serverless functionality since serverless doesn't have data tiers.
Additional information
resolves #167741