In the future we want to rethink index pattern, and one part of index patterns are scripted fields. You can basically specify a Painless script that will be executed instead of a field. So basically that script you specify in index pattern management is inlined into all requests.
I think we want to move that more towards the editor (as nearly everything in index patterns). Also there are a couple of common "scripts" users might want to easily execute without needing to write painless. Let me give you an example: A user might have a field that's looking like 12:18:00 as a string. We don't only want to allow the user to use that field as it is, but also give them the capabilities to e.g. "split that field" by the colon and use then the first one. Splitting fields by special characters is common, so I think we might want to have an option for that in the UI. Behind the scenes that would generate the script for that (return doc['fieldname'].value.split(':')[1] (simplified)) and use that instead of the field (meaning we also must make sure esaggs and esdocs will understand scripts directly instead of fields. We could have common options (like split a field, search and replace a field value, etc.) and have sophisticated safe painless scripts for that as templates that we use behind the scenes, thus the user doesn't need to write a complex painless script (because that version was very simplified, and a sophisticated would have null checks, etc.).
Besides having those scripted templates we also might allow the user to add a custom script directly via the fieldlist (not via the index pattern), since that would allow us to more easily get rid of index patterns later on. We should rather have a way to store those scripts in some kind of library and make them easily reusable later (on any index pattern).
Tags: derived fields / scripted fields / calculated fields
Implementation notes
The first integration will be done using runtime fields saved in the index pattern. Lens provides a way to create new fields and delete/edit existing fields without leaving Lens (using the integration provided here: #88995 ). In a later phase fields saved as part of the Visualization can get added.
In the future we want to rethink index pattern, and one part of index patterns are scripted fields. You can basically specify a Painless script that will be executed instead of a field. So basically that script you specify in index pattern management is inlined into all requests.
I think we want to move that more towards the editor (as nearly everything in index patterns). Also there are a couple of common "scripts" users might want to easily execute without needing to write painless. Let me give you an example: A user might have a field that's looking like
12:18:00as a string. We don't only want to allow the user to use that field as it is, but also give them the capabilities to e.g. "split that field" by the colon and use then the first one. Splitting fields by special characters is common, so I think we might want to have an option for that in the UI. Behind the scenes that would generate the script for that (return doc['fieldname'].value.split(':')[1](simplified)) and use that instead of the field (meaning we also must make sureesaggsandesdocswill understand scripts directly instead of fields. We could have common options (like split a field, search and replace a field value, etc.) and have sophisticated safe painless scripts for that as templates that we use behind the scenes, thus the user doesn't need to write a complex painless script (because that version was very simplified, and a sophisticated would havenullchecks, etc.).Besides having those scripted templates we also might allow the user to add a custom script directly via the fieldlist (not via the index pattern), since that would allow us to more easily get rid of index patterns later on. We should rather have a way to store those scripts in some kind of library and make them easily reusable later (on any index pattern).
Tags: derived fields / scripted fields / calculated fields
Implementation notes
The first integration will be done using runtime fields saved in the index pattern. Lens provides a way to create new fields and delete/edit existing fields without leaving Lens (using the integration provided here: #88995 ). In a later phase fields saved as part of the Visualization can get added.