Skip to content

Commit 845da55

Browse files
Wylie Conlondej611kibanamachineflash1293MichaelMarcialis
committed
[Lens] Formula editor (#99297)
* 💄 Hack to fix suggestion box * 🐛 Fix validation messages * 🐛 Relax operations check for managedReferences * Change completion params * 🏷️ Fix missing arg issue * ✨ Add more tinymath fns * 🐛 Improved validation around math operations + multiple named arguments * 🐛 Use new onError feature in math expression * ♻️ Refactor namedArguments validation * 🐛 Fix circular dependency issue in tests + minor fixes * Move formula into a tab * 🔥 Leftovers from previous merge * ✨ Move over namedArgs from previous function * ✅ Add tests for transferable scenarios * ✅ Fixed broken test * ✨ Use custom label for axis * Allow switching back and forth to formula tab * Add a section for the function reference * Add modal editor and markdown docs * Change the way math nodes are validated * Use custom portal to fix monaco positioning * Fix model sharing issues * Provide signature help * 🐛 Fix small test issue * 🐛 Mark pow arguments as required * 🐛 validate on first render only if a formula is present * 🔥 Remove log10 fn for now * ✨ Improved math validation + add tests for math functions * Fix mount/unmount issues with Monaco * [Lens] Fully unmount React when flyout closes * Fix bug with editor frame unmounting * Fix type * Add tests for monaco providers, add hover provider * Add test for last_value * Usability improvements * Add KQL and Lucene named parameters * Add kql, lucene completion and validation * Fix autocomplete on weird characters and properly connect KQL * Highlight functions that have additional requirements after validating * Fix type error and move help text to popover * Fix escape characters inside KQL * 🐛 Fix dataType issue when moving over to Formula * Automatically insert single quotes on every named param * Only insert single quotes when typing kql= or lucene= * Reorganize help popover * Fix merge issues * Update grammar for formulas * Fix bad merge * Rough fullscreen mode * Type updates * Pass through fullscreen state * Remove more chrome from full screen mode * Fix minor bugs in formula typing * 🐛 Decouple column order of references and output * 🔧 Fix tests and types * ✅ Add first functional test * Fix copying formulas and empty formula * Trigger suggestion prompt when hitting enter on function or typing kql= * 🐛 Prevent flyout from closing while interacting with monaco * refactoring * move main column generation into parse module * fix tests * refactor small formula styles and markup * documentation * adjustments in formula footer * Formula refactoring (#12) * refactoring * move main column generation into parse module * fix tests * more style and markup tweak for custom formula * Fix tests * [Expressions] Use table column ID instead of name when set * [Lens] Create managedReference type for formulas * Fix test failures * Fix i18n types * fix fullscreen flex issues * Delete managedReference when replacing * refactor css and markup; add button placeholders * [Lens] Formulas * Tests for formula Co-authored-by: Marco Liberati <marco.liberati@elastic.co> * added error count placeholder * Add tooltips * Refactoring from code review * Fix some editor issues * Update ID matching to match by name sometimes * Improve performance of Monaco, fix formulas with 0, update labels * Improve performance of full screen toggle * Fix formula tests * fix stuff * Add an extra case to prevent insertion of duplicate column * Simplify logic and add test for output ID * add telemetry for Lens formula (#15) * Respond to review comments * ✨ Improve the signatures with better documentation and examples * adjust border styles to account for docs collapse * refactor docs markup; restructure docs obj; styles * Fix formula auto reordering (#18) * fix formula auto reordering * add unit test * Fix and improve suggestion experience in Formula (#19) * ✨ Revisit documentation and suggestions * 👌 Integrated feedback * ✨ Add query validation for quotes * Usability updates & type fixes * add search to formula * fix form styles to match designs * fix text styles; revert to Markdown for control * 👌 Integrated more feedback * improve search * improve suggestions * improve suggestions even more * 🐛 Fix i18n issues (#22) * Persist formula on leave, fix fullscreen and popovers * Fix documentation tests * 🏷️ fix type issue * 🐛 Remove hidden operations from valid functions list * 🐛 Fix empty string query edge case * 🐛 Enable more suggestions + extends validation * Fix tests that depended on setState being called without function * Error state and text wrapping updates * ✨ Add new module to CodeEditor for brackets matching (#25) * Fix type * show warning * keep current quick function * ✨ Improve suggestions within kql query * 📷 Fix snapshot editor test * 🐛 Improved suggestion for single quote and refactored debounce * Fix lodash usage * Fix tests * Revert "keep current quick function" This reverts commit ed47705. * Improve performance of dispatch by using timeout * Improve memoization of datapanel * Fix escape characters * fix reduced suggestions * fix responsiveness * fix unit test * Fix autocomplete on nested math * Show errors and warnings on first render * fix transposing column crash * Update comment * 🐛 Fix field error message * fix test types * 📝 Fix i18n name * 💄 Manage wordwrap via react component * Fix selector for palettes that interferes with quick functions * Use word wrapping by default * Errors for managed references are handled at the top level * 🐛 Move the cursor just next to new inserted text * ⚗️ First pass for performance * 🐛 Fix unwanted change * ⚡ Memoize as many combobox props as possible * ⚡ More memoization * Show errors in hover * Use temporary invalid state when moving away from formula * Remove setActiveDimension and shouldClose, fixed by async setters * Fix test dependency * do not show quick functions tab * increase documentation popover width * fix functional test * Call setActiveDimension when updating visualization * Simplify handling of flyout with incomplete columns * Fix test issues * add description to formula telemetry * fix schema * Update from design feedback * More review comments * Hide callout border from v7 theme Co-authored-by: dej611 <dej611@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Joe Reuter <johannes.reuter@elastic.co> Co-authored-by: Michael Marcialis <michael.marcialis@elastic.co> Co-authored-by: Joe Reuter <email@johannes-reuter.de> Co-authored-by: Marco Liberati <marco.liberati@elastic.co> Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
1 parent 59b05c4 commit 845da55

84 files changed

Lines changed: 5117 additions & 659 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/kbn-monaco/src/monaco_imports.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,5 +21,6 @@ import 'monaco-editor/esm/vs/editor/contrib/folding/folding.js'; // Needed for f
2121
import 'monaco-editor/esm/vs/editor/contrib/suggest/suggestController.js'; // Needed for suggestions
2222
import 'monaco-editor/esm/vs/editor/contrib/hover/hover.js'; // Needed for hover
2323
import 'monaco-editor/esm/vs/editor/contrib/parameterHints/parameterHints.js'; // Needed for signature
24+
import 'monaco-editor/esm/vs/editor/contrib/bracketMatching/bracketMatching.js'; // Needed for brackets matching highlight
2425

2526
export { monaco };

‎packages/kbn-tinymath/grammar/grammar.peggy‎

Lines changed: 26 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
// tinymath parsing grammar
22

33
{
4-
function simpleLocation (location) {
5-
// Returns an object representing the position of the function within the expression,
6-
// demarcated by the position of its first character and last character. We calculate these values
7-
// using the offset because the expression could span multiple lines, and we don't want to deal
8-
// with column and line values.
9-
return {
10-
min: location.start.offset,
11-
max: location.end.offset
4+
function simpleLocation (location) {
5+
// Returns an object representing the position of the function within the expression,
6+
// demarcated by the position of its first character and last character. We calculate these values
7+
// using the offset because the expression could span multiple lines, and we don't want to deal
8+
// with column and line values.
9+
return {
10+
min: location.start.offset,
11+
max: location.end.offset
12+
}
1213
}
13-
}
1414
}
1515

1616
start
@@ -74,26 +74,34 @@ Expression
7474
= AddSubtract
7575

7676
AddSubtract
77-
= _ left:MultiplyDivide rest:(('+' / '-') MultiplyDivide)* _ {
78-
return rest.reduce((acc, curr) => ({
77+
= _ left:MultiplyDivide rest:(('+' / '-') MultiplyDivide)+ _ {
78+
const topLevel = rest.reduce((acc, curr) => ({
7979
type: 'function',
8080
name: curr[0] === '+' ? 'add' : 'subtract',
8181
args: [acc, curr[1]],
82-
location: simpleLocation(location()),
83-
text: text()
84-
}), left)
82+
}), left);
83+
if (typeof topLevel === 'object') {
84+
topLevel.location = simpleLocation(location());
85+
topLevel.text = text();
86+
}
87+
return topLevel;
8588
}
89+
/ MultiplyDivide
8690

8791
MultiplyDivide
8892
= _ left:Factor rest:(('*' / '/') Factor)* _ {
89-
return rest.reduce((acc, curr) => ({
93+
const topLevel = rest.reduce((acc, curr) => ({
9094
type: 'function',
9195
name: curr[0] === '*' ? 'multiply' : 'divide',
9296
args: [acc, curr[1]],
93-
location: simpleLocation(location()),
94-
text: text()
95-
}), left)
97+
}), left);
98+
if (typeof topLevel === 'object') {
99+
topLevel.location = simpleLocation(location());
100+
topLevel.text = text();
101+
}
102+
return topLevel;
96103
}
104+
/ Factor
97105

98106
Factor
99107
= Group

‎packages/kbn-tinymath/index.d.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,11 @@ export interface TinymathLocation {
2424
export interface TinymathFunction {
2525
type: 'function';
2626
name: string;
27-
text: string;
2827
args: TinymathAST[];
29-
location: TinymathLocation;
28+
// Location is not guaranteed because PEG grammars are not left-recursive
29+
location?: TinymathLocation;
30+
// Text is not guaranteed because PEG grammars are not left-recursive
31+
text?: string;
3032
}
3133

3234
export interface TinymathVariable {

‎packages/kbn-tinymath/test/library.test.js‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,35 @@ describe('Parser', () => {
4141
});
4242
});
4343

44+
describe('Math', () => {
45+
it('converts basic symbols into left-to-right pairs', () => {
46+
expect(parse('a + b + c - d')).toEqual({
47+
args: [
48+
{
49+
name: 'add',
50+
type: 'function',
51+
args: [
52+
{
53+
name: 'add',
54+
type: 'function',
55+
args: [
56+
expect.objectContaining({ location: { min: 0, max: 2 } }),
57+
expect.objectContaining({ location: { min: 3, max: 6 } }),
58+
],
59+
},
60+
expect.objectContaining({ location: { min: 7, max: 10 } }),
61+
],
62+
},
63+
expect.objectContaining({ location: { min: 11, max: 13 } }),
64+
],
65+
name: 'subtract',
66+
type: 'function',
67+
text: 'a + b + c - d',
68+
location: { min: 0, max: 13 },
69+
});
70+
});
71+
});
72+
4473
describe('Variables', () => {
4574
it('strings', () => {
4675
expect(parse('f')).toEqual(variableEqual('f'));
@@ -263,6 +292,8 @@ describe('Evaluate', () => {
263292
expect(evaluate('5/20')).toEqual(0.25);
264293
expect(evaluate('1 + 1 + 2 + 3 + 12')).toEqual(19);
265294
expect(evaluate('100 / 10 / 10')).toEqual(1);
295+
expect(evaluate('0 * 1 - 100 / 10 / 10')).toEqual(-1);
296+
expect(evaluate('100 / (10 / 10)')).toEqual(100);
266297
});
267298

268299
it('equations with functions', () => {

‎src/plugins/dashboard/server/usage/dashboard_telemetry.test.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,22 @@ const lensXYSeriesB = ({
7272
visualization: {
7373
preferredSeriesType: 'seriesB',
7474
},
75+
datasourceStates: {
76+
indexpattern: {
77+
layers: {
78+
first: {
79+
columns: {
80+
first: {
81+
operationType: 'terms',
82+
},
83+
second: {
84+
operationType: 'formula',
85+
},
86+
},
87+
},
88+
},
89+
},
90+
},
7591
},
7692
},
7793
},
@@ -144,6 +160,7 @@ describe('dashboard telemetry', () => {
144160
expect(collectorData.lensByValue.a).toBe(3);
145161
expect(collectorData.lensByValue.seriesA).toBe(2);
146162
expect(collectorData.lensByValue.seriesB).toBe(1);
163+
expect(collectorData.lensByValue.formula).toBe(1);
147164
});
148165

149166
it('handles misshapen lens panels', () => {

‎src/plugins/dashboard/server/usage/dashboard_telemetry.ts‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,16 @@ interface LensPanel extends SavedDashboardPanel730ToLatest {
2727
visualization?: {
2828
preferredSeriesType?: string;
2929
};
30+
datasourceStates?: {
31+
indexpattern?: {
32+
layers: Record<
33+
string,
34+
{
35+
columns: Record<string, { operationType: string }>;
36+
}
37+
>;
38+
};
39+
};
3040
};
3141
};
3242
};
@@ -109,6 +119,19 @@ export const collectByValueLensInfo: DashboardCollectorFunction = (panels, colle
109119
}
110120

111121
collectorData.lensByValue[type] = collectorData.lensByValue[type] + 1;
122+
123+
const hasFormula = Object.values(
124+
lensPanel.embeddableConfig.attributes.state?.datasourceStates?.indexpattern?.layers || {}
125+
).some((layer) =>
126+
Object.values(layer.columns).some((column) => column.operationType === 'formula')
127+
);
128+
129+
if (hasFormula && !collectorData.lensByValue.formula) {
130+
collectorData.lensByValue.formula = 0;
131+
}
132+
if (hasFormula) {
133+
collectorData.lensByValue.formula++;
134+
}
112135
}
113136
}
114137
};

‎src/plugins/expressions/common/expression_functions/specs/map_column.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import { Observable } from 'rxjs';
1010
import { take } from 'rxjs/operators';
1111
import { i18n } from '@kbn/i18n';
1212
import { ExpressionFunctionDefinition } from '../types';
13-
import { Datatable, getType } from '../../expression_types';
13+
import { Datatable, DatatableColumn, getType } from '../../expression_types';
1414

1515
export interface MapColumnArguments {
1616
id?: string | null;
@@ -110,10 +110,10 @@ export const mapColumn: ExpressionFunctionDefinition<
110110

111111
return Promise.all(rowPromises).then((rows) => {
112112
const type = rows.length ? getType(rows[0][columnId]) : 'null';
113-
const newColumn = {
113+
const newColumn: DatatableColumn = {
114114
id: columnId,
115115
name: args.name,
116-
meta: { type },
116+
meta: { type, params: { id: type } },
117117
};
118118
if (args.copyMetaFrom) {
119119
const metaSourceFrom = columns.find(({ id }) => id === args.copyMetaFrom);

‎src/plugins/expressions/common/expression_functions/specs/tests/map_column.test.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,11 @@ describe('mapColumn', () => {
2929
expect(result.type).toBe('datatable');
3030
expect(result.columns).toEqual([
3131
...testTable.columns,
32-
{ id: 'pricePlusTwo', name: 'pricePlusTwo', meta: { type: 'number' } },
32+
{
33+
id: 'pricePlusTwo',
34+
name: 'pricePlusTwo',
35+
meta: { type: 'number', params: { id: 'number' } },
36+
},
3337
]);
3438
expect(result.columns[result.columns.length - 1]).toHaveProperty('name', 'pricePlusTwo');
3539
expect(result.rows[arbitraryRowIndex]).toHaveProperty('pricePlusTwo');

‎src/plugins/kibana_react/public/code_editor/__snapshots__/code_editor.test.tsx.snap‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎src/plugins/kibana_react/public/code_editor/code_editor.tsx‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -187,10 +187,16 @@ export class CodeEditor extends React.Component<Props, {}> {
187187
wordBasedSuggestions: false,
188188
wordWrap: 'on',
189189
wrappingIndent: 'indent',
190+
matchBrackets: 'never',
190191
...options,
191192
}}
192193
/>
193-
<ReactResizeDetector handleWidth handleHeight onResize={this._updateDimensions} />
194+
<ReactResizeDetector
195+
handleWidth
196+
handleHeight
197+
onResize={this._updateDimensions}
198+
refreshMode="debounce"
199+
/>
194200
</>
195201
);
196202
}

0 commit comments

Comments
 (0)