Skip to content

[PoC] lens esql editor#245716

Draft
walterra wants to merge 24 commits intoelastic:mainfrom
walterra:lens-esql-poc-3
Draft

[PoC] lens esql editor#245716
walterra wants to merge 24 commits intoelastic:mainfrom
walterra:lens-esql-poc-3

Conversation

@walterra
Copy link
Copy Markdown
Contributor

@walterra walterra commented Dec 9, 2025 •
edited
Loading

Summary

New rebase of #199632.

This branch implements a proof-of-concept feature that allows converting Lens visualizations from the form-based datasource to ES|QL queries.

lens-ux-esql-conversion-0001

ES|QL Generation

  • Converts form-based Lens configurations (aggregations, buckets, metrics) into equivalent ES|QL queries - Implemented in to_esql.ts using the @kbn/esql-composer library
  • Generates properly structured ES|QL with:
    • FROM clause for index patterns
    • WHERE filters for time ranges and conditions
    • STATS aggregations for metrics and groupings
    • SORT clauses for bucket ordering

Limitations (these will prevent conversion):

  • Non-UTC timezones (utcOffset !== 0)
  • Formula operations
  • Time shift configurations
  • Runtime fields
  • Reduced time range on metrics without date histogram

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
  • Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Dec 9, 2025

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!
  • Click to trigger kibana-entity-store-performance-from-pr for this PR!

@walterra walterra self-assigned this Dec 22, 2025
@walterra walterra mentioned this pull request Jan 8, 2026
Merged
4 tasks
walterra added a commit that referenced this pull request Jan 29, 2026
@walterra @stratoula
[Lens] [ES|QL] Single layer ES|QL conversion (#248078)
e0bb388 
## Summary

Follow up to #246209. Implements
elastic/kibana-team#2391. Also implements part
of elastic/kibana-team#2179.

This applies the work done in PoCs #199632 and #245716 to the new UI
flow with the conversion modal added in #246209.

This PR wires up the ES|QL conversion modal to actually perform the
datasource conversion. Related to that, it also cleans up the inline
editing flyout code by removing redundant props.

- `handleConvertToEsql` callback to switch flyout from formBased to
textBased mode
- Updates visualization state to remap column IDs (xAccessor, accessors,
splitAccessor) to ES|QL field names
- Reduces prop drilling of `datasourceId` and `canEditTextBasedQuery`,
adds `hideTextBasedEditor` to Redux state for Discover integration.

Note: The full feature is still behind a dev-mode flag, it's not enabled
in production. The scope of this PR is to enable the full UI flow of
DSL->ES|QL conversion and enabling the conversion based on wether the
button to convert is enabled. Out of scope of the PR is fine tuning
actual conversions, we'll pick that up in follow-ups.


https://github.com/user-attachments/assets/d545c759-417b-4705-a820-bc669ca17ae2

### To test that the conversion button isn't showing up in production
builds:

```
# local kibana build
node scripts/build --skip-os-packages --skip-docker-contexts

# create ./build/default/kibana-9.4.0-SNAPSHOT-darwin-aarch64/config/kibana.yml
elasticsearch.hosts:
  - https://localhost:9200/
elasticsearch.password: ****
elasticsearch.username: kibana_system

# run kibana
./build/default/kibana-9.4.0-SNAPSHOT-darwin-aarch64/bin/kibana --elasticsearch.ssl.verificationMode=none
```

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.
- [x] Used `cursor-cli` with `opus-4.5` to refactor the PoCs onto the
new UI flow.

---------

Co-authored-by: Stratou <efstratia.kalafateli@elastic.co>
hannahbrooks pushed a commit to hannahbrooks/kibana that referenced this pull request Jan 30, 2026
@walterra @stratoula @hannahbrooks
[Lens] [ES|QL] Single layer ES|QL conversion (elastic#248078)
6997609 
## Summary

Follow up to elastic#246209. Implements
elastic/kibana-team#2391. Also implements part
of elastic/kibana-team#2179.

This applies the work done in PoCs elastic#199632 and elastic#245716 to the new UI
flow with the conversion modal added in elastic#246209.

This PR wires up the ES|QL conversion modal to actually perform the
datasource conversion. Related to that, it also cleans up the inline
editing flyout code by removing redundant props.

- `handleConvertToEsql` callback to switch flyout from formBased to
textBased mode
- Updates visualization state to remap column IDs (xAccessor, accessors,
splitAccessor) to ES|QL field names
- Reduces prop drilling of `datasourceId` and `canEditTextBasedQuery`,
adds `hideTextBasedEditor` to Redux state for Discover integration.

Note: The full feature is still behind a dev-mode flag, it's not enabled
in production. The scope of this PR is to enable the full UI flow of
DSL->ES|QL conversion and enabling the conversion based on wether the
button to convert is enabled. Out of scope of the PR is fine tuning
actual conversions, we'll pick that up in follow-ups.


https://github.com/user-attachments/assets/d545c759-417b-4705-a820-bc669ca17ae2

### To test that the conversion button isn't showing up in production
builds:

```
# local kibana build
node scripts/build --skip-os-packages --skip-docker-contexts

# create ./build/default/kibana-9.4.0-SNAPSHOT-darwin-aarch64/config/kibana.yml
elasticsearch.hosts:
  - https://localhost:9200/
elasticsearch.password: ****
elasticsearch.username: kibana_system

# run kibana
./build/default/kibana-9.4.0-SNAPSHOT-darwin-aarch64/bin/kibana --elasticsearch.ssl.verificationMode=none
```

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.
- [x] Used `cursor-cli` with `opus-4.5` to refactor the PoCs onto the
new UI flow.

---------

Co-authored-by: Stratou <efstratia.kalafateli@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@walterra walterra

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@walterra @elasticmachine @ppisljar