Skip to content

[Console Monaco] Resolve uncaught error from tokenizer#188746

Merged
ElenaStoeva merged 3 commits intoelastic:mainfrom
ElenaStoeva:console/resolve-tokenizer-exception
Jul 23, 2024
Merged

[Console Monaco] Resolve uncaught error from tokenizer#188746
ElenaStoeva merged 3 commits intoelastic:mainfrom
ElenaStoeva:console/resolve-tokenizer-exception

Conversation

@ElenaStoeva
Copy link
Copy Markdown
Contributor

@ElenaStoeva ElenaStoeva commented Jul 19, 2024 •
edited
Loading

Fixes #186765

Summary

This PR fixes the uncaught error from the tokenizer when we add the input below:

Test input 
GET /
GET _cluster/settings?include_defaults&flat_settings&filter_path=*.*interval*
PUT _cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "10s"
  }
}

GET _cat/${TEST}

PUT _data_stream/test-cold
POST test-cold/_rollover

GET _cat/indices?v&h=
GET _cat/indices/.alerts-observability.threshold.alerts-default,.alerts-default.alerts-default,

GET _cat/indices,aliases

GET _cat/shards/test*?v
"""
index                                    shard prirep state      docs store dataset ip         node
.ds-test-cold-2024.05.20-000002          0     p      STARTED       0  249b    249b 172.19.0.2 839a1677bdab
.ds-test-cold-2024.05.20-000002          0     r      UNASSIGNED                               
restored-.ds-test-cold-2024.05.20-000001 0     p      STARTED       0  289b    289b 172.19.0.2 839a1677bdab
"""

GET restored-.ds-test-cold-2024.05.20-000001/_settings

"""
          "snapshot": {
            "snapshot_name": "2024.05.20-.ds-test-cold-2024.05.20-000001-test-h5yr-_tws6es6biug0uqbw",
            "index_uuid": "8tXgFf_cR2KIMofpj2I7Uw",
            "repository_uuid": "3i3oPavIQZGeYOQzRZajyQ",
            "index_name": ".ds-test-cold-2024.05.20-000001",
            "repository_name": "fs",
            "snapshot_uuid": "3S3CDGZgSh2XQSRO3UTxVA"
          }
"""

DELETE _snapshot/fs/2024.05.20-.ds-test-cold-2024.05.20-000001-test-h5yr-_tws6es6biug0uqbw

POST _ilm/stop

POST restored-.ds-test-cold-2024.05.20-000001/_clone/.ds-test-cold-2024.05.20-000001
{
  "settings": {
    "index.store.type": null,
    "index.recovery.type": null
  }
}

# Check the cloned index & shards
GET _cat/indices/.ds-test-cold-2024.05.20-000001?v
GET _cat/shards/.ds-test-cold-2024.05.20-000001?v

PUT _snapshot/fs/2024.05.20-.ds-test-cold-2024.05.20-000001
{
  "indices": ".ds-test-cold-2024.05.20-000001",
  "ignore_unavailable": false,
  "include_global_state": false
}

# Check the taken snapshot

GET _snapshot/fs/2024.05.20-.ds-test-cold-2024.05.20-000001
"""
      "snapshot": "2024.05.20-.ds-test-cold-2024.05.20-000001",
      "uuid": "GC7xoG1yQN6W5LIUMqCDYA",
"""




PUT _snapshot/fs/temp-snapshot-for-recovery
{
  "indices": "restored-.ds-test-cold-2024.05.20-000001",
  "ignore_unavailable": false,
  "include_global_state": false
}
GET _snapshot/fs/temp-snapshot-for-recovery
"""
      "snapshot": "temp-snapshot-for-recovery",
      "uuid": "NgFDtQMESv2o4BZd0dsP8g",
"""

GET .ds-test-cold-2024.05.20-000002
"""
{
  ".ds-test-cold-2024.05.20-000002": {
    "aliases": {},
    "mappings": {
      "_data_stream_timestamp": {
        "enabled": true
      },
      "properties": {
        "@timestamp": {
          "type": "date"
        }
      }
    },
    "settings": {
      "index": {
        "lifecycle": {
          "name": "test"
        },
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_hot"
            }
          }
        },
        "hidden": "true",
        "number_of_shards": "1",
        "provided_name": ".ds-test-cold-2024.05.20-000002",
        "creation_date": "1716175738940",
        "priority": "100",
        "number_of_replicas": "1",
        "uuid": "nvldBwt2QxeQzpkrbK69qQ",
        "version": {
          "created": "8503000"
        }
      }
    },
    "data_stream": "test-cold"
  }
}
"""

How to test: Add the input above in Console editor and verify that there is no uncaught error in the browser error.

This change also resolves some highlighting inconsistencies, where some text was incorrectly highlighted as a method and url tokens:

Before:

Screenshot 2024-07-19 at 17 15 05 Screenshot 2024-07-19 at 17 24 33



Now:

Screenshot 2024-07-19 at 17 12 07 Screenshot 2024-07-19 at 17 25 39

@ElenaStoeva ElenaStoeva added Feature:Console Dev Tools Console Feature Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// release_note:skip Skip the PR/issue when compiling release notes labels Jul 19, 2024
@ElenaStoeva ElenaStoeva requested a review from yuliacech July 19, 2024 13:05
@ElenaStoeva ElenaStoeva self-assigned this Jul 19, 2024
@ElenaStoeva
Copy link
Copy Markdown
Contributor Author

ElenaStoeva commented Jul 19, 2024

/ci

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 19, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #6 / Template fields renders template fields with existing value

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-srcJs 3.2MB 3.2MB +106.0B

cc @ElenaStoeva

@ElenaStoeva ElenaStoeva marked this pull request as ready for review July 19, 2024 16:27
@ElenaStoeva ElenaStoeva requested a review from a team as a code owner July 19, 2024 16:27
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 19, 2024

Pinging @elastic/kibana-management (Team:Kibana Management)

yuliacech
yuliacech reviewed Jul 23, 2024
View reviewed changes
packages/kbn-monaco/src/console/lexer_rules/console_editor.ts Outdated
matchTokensWithEOL('method', /([a-zA-Z]+)/, 'root', 'method_sep'),
matchTokensWithEOL(
'method',
/([Gg][Ee][Tt])|([Pp][Oo][Ss][Tt])|([Pp][Uu][Tt])|([Pp][Aa][Tt][Cc][Hh])|([Dd][Ee][Ll][Ee][Tt][Ee])|([Hh][Ee][Aa][Dd])/,
Copy link
Copy Markdown
Contributor

@yuliacech yuliacech Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) this regex is somewhat harder to read: we could either put it in a named variable or change the regex to /get|post|put|patch|delete|head/ and add the option ignoreCase: true to the rules

Copy link
Copy Markdown
Contributor Author

@ElenaStoeva ElenaStoeva Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good call! I wasn't sure if adding ignoreCase: true wouldn't affect the rest of the rules but now I checked and it seems none of the rules rely on strict case sensitivity so it should work. I added this change with 95e7d93.

yuliacech
yuliacech approved these changes Jul 23, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@yuliacech yuliacech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a lot for fixing this error, @ElenaStoeva!

@ElenaStoeva ElenaStoeva enabled auto-merge (squash) July 23, 2024 16:19
@kibana-ci
Copy link
Copy Markdown

kibana-ci commented Jul 23, 2024

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-srcJs 3.3MB 3.3MB +33.0B

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ElenaStoeva

@ElenaStoeva ElenaStoeva merged commit db5a839 into elastic:main Jul 23, 2024
@kibanamachine kibanamachine added v8.16.0 backport:skip This PR does not require backporting labels Jul 23, 2024
TinLe added a commit to TinLe/kibana that referenced this pull request Jul 30, 2024
@TinLe
Merge branch 'master' into dependabot/npm_and_yarn/express-4.19.2
df656a6 
* master: (3487 commits)
  `BedrockChat` & `GeminiChat` (elastic#186809)
  [ResponseOps] log error when ES Query rules find docs out of time range (elastic#186332)
  skip flaky suite (elastic#188997)
  [Security solution][Alert Details] Enable preview feature flag and cypress tests (elastic#188580)
  [EuiProviders] Warn Developer if EuiProvider is missing (elastic#184608)
  [Security Solution ] Fixes Timeline infinite loading bug (elastic#188943)
  Improve SearchSource SearchRequest type (elastic#186862)
  Deprecate Search Sessions config (elastic#188037)
  [Synthetics] Add missing monitorType and tag info in cards !! (elastic#188824)
  [Console Monaco] Resolve uncaught error from tokenizer (elastic#188746)
  [Data Forge] Add `service.logs` dataset as a  data stream (elastic#188786)
  [Console] Fix failing bulk requests (elastic#188552)
  Update dependency terser to ^5.31.2 (main) (elastic#188528)
  [APM][ECO] Telemetry (elastic#188627)
  [Fleet] Fix uninstall package validation accross space (elastic#188749)
  Update warning on `xpack.fleet.enableExperimental` (elastic#188917)
  [DOCS][Cases] Automate more screenshots for cases (elastic#188697)
  [Fleet] Fix get one agent when feature flag disabled (elastic#188953)
  chore(investigate): Add investigate-app plugin from poc (elastic#188122)
  [Monaco Editor] Add Search functionality (elastic#188337)
  ...
TinLe added a commit to TinLe/kibana that referenced this pull request Jul 30, 2024
@TinLe
Merge branch 'master' into dependabot/npm_and_yarn/pdfjs-dist-4.2.67
8bec40b 
* master: (2400 commits)
  `BedrockChat` & `GeminiChat` (elastic#186809)
  [ResponseOps] log error when ES Query rules find docs out of time range (elastic#186332)
  skip flaky suite (elastic#188997)
  [Security solution][Alert Details] Enable preview feature flag and cypress tests (elastic#188580)
  [EuiProviders] Warn Developer if EuiProvider is missing (elastic#184608)
  [Security Solution ] Fixes Timeline infinite loading bug (elastic#188943)
  Improve SearchSource SearchRequest type (elastic#186862)
  Deprecate Search Sessions config (elastic#188037)
  [Synthetics] Add missing monitorType and tag info in cards !! (elastic#188824)
  [Console Monaco] Resolve uncaught error from tokenizer (elastic#188746)
  [Data Forge] Add `service.logs` dataset as a  data stream (elastic#188786)
  [Console] Fix failing bulk requests (elastic#188552)
  Update dependency terser to ^5.31.2 (main) (elastic#188528)
  [APM][ECO] Telemetry (elastic#188627)
  [Fleet] Fix uninstall package validation accross space (elastic#188749)
  Update warning on `xpack.fleet.enableExperimental` (elastic#188917)
  [DOCS][Cases] Automate more screenshots for cases (elastic#188697)
  [Fleet] Fix get one agent when feature flag disabled (elastic#188953)
  chore(investigate): Add investigate-app plugin from poc (elastic#188122)
  [Monaco Editor] Add Search functionality (elastic#188337)
  ...
@ElenaStoeva ElenaStoeva deleted the console/resolve-tokenizer-exception branch January 31, 2026 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@sebelga sebelga sebelga approved these changes

@yuliacech yuliacech yuliacech approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@ElenaStoeva ElenaStoeva

Labels

backport:skip This PR does not require backporting Feature:Console Dev Tools Console Feature release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// v8.16.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Console][Monaco] Uncaught Error: console: maximum tokenizer stack size reached: [json_root,root,...]

6 participants

@ElenaStoeva @elasticmachine @kibana-ci @sebelga @yuliacech @kibanamachine