Skip to content

Enable fallback synthetic source for token_count#109044

Merged
lkts merged 4 commits intoelastic:mainfrom
lkts:feature/more_synthetic_source_fallback
May 27, 2024
Merged

Enable fallback synthetic source for token_count#109044
lkts merged 4 commits intoelastic:mainfrom
lkts:feature/more_synthetic_source_fallback

Conversation

@lkts
Copy link
Copy Markdown
Contributor

@lkts lkts commented May 24, 2024

Enable fallback synthetic source for token_count and add tests.

@lkts lkts added >feature :StorageEngine/Mapping The storage related side of mappings labels May 24, 2024
@github-actions
Copy link
Copy Markdown
Contributor

Documentation preview:

@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Hi @lkts, I've created a changelog YAML for you.


@Override
public BlockLoader blockLoader(BlockLoaderContext blContext) {
return null;
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

token_count is not supported in ESQL but TokenCountFieldType inherits blockLoader implementation from NumberFieldType which leads to fun test failures. IMO it is more logical to do this until it is actually supported.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that was not a good idea it seems.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are okay, i was just confused.

Copy link
Copy Markdown
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - Typically token_count field type is configured as a multi-field, so it isn't used as part of synthetic source. This change makes it possible to synthesize source in the case a token_count field is configured a main field.

@martijnvg
Copy link
Copy Markdown
Member

@elasticmachine run elasticsearch-ci/part-4

@lkts
Copy link
Copy Markdown
Contributor Author

lkts commented May 27, 2024

@elasticmachine update branch

@lkts lkts merged commit 42f4294 into elastic:main May 27, 2024
@lkts lkts deleted the feature/more_synthetic_source_fallback branch May 27, 2024 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants