Skip to content

SearchLookup to not require MapperService as a constructor argument#64017

Merged
javanna merged 2 commits intoelastic:masterfrom
javanna:refactoring/search_lookup_mapper_service
Oct 22, 2020
Merged

SearchLookup to not require MapperService as a constructor argument#64017
javanna merged 2 commits intoelastic:masterfrom
javanna:refactoring/search_lookup_mapper_service

Conversation

@javanna
Copy link
Copy Markdown
Contributor

@javanna javanna commented Oct 21, 2020

SearchLookup currently takes the whole MapperService as an argument, and it passes it over to DocLookup and FieldsLookup. In practice, what they all need is only a way to lookup field types given a field name.

With this commit, SearchLookup requires a Function<String,MappedFieldType> to be created rather than the whole MapperService. This helps clarifying the dependency between MapperService and SearchLookup, especially as we introduce runtime fields defined in the search request: they can be exposed at QueryShardContext level, through its lookup methods, rather than at the MapperService level which QueryShardContext depends on.

SearchLookup currently takes the whole MapperService as an argument, and it passes it over to DocLookup and FieldsLookup. In practice, what they all need is only a way to lookup field types given a field name.

With this commit, SearchLookup requires a Function<String,MappedFieldType> to be created rather than the whole MapperService. This helps clarifying the dependency between MapperService and SearchLookup, especially as we introduce runtime fields defined in the search request: they can be exposed at QueryShardContext level, through its lookup methods, rather than at the MapperService level which QueryShardContext depends on.
@javanna javanna added :Search Foundations/Mapping Index mappings, including merging and defining field types >refactoring v8.0.0 v7.11.0 labels Oct 21, 2020
@javanna javanna requested a review from nik9000 October 21, 2020 20:00
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-search (:Search/Mapping)

@elasticmachine elasticmachine added the Team:Search Meta label for search team label Oct 21, 2020
@javanna javanna requested a review from romseygeek October 21, 2020 20:05
@nik9000
Copy link
Copy Markdown
Member

nik9000 commented Oct 21, 2020

SearchLookup requires a Function<String,MappedFieldType> to be created rather than the whole MapperService

This sounds like it could make a lot of tests a lot shorter!

@javanna
Copy link
Copy Markdown
Contributor Author

javanna commented Oct 21, 2020

This sounds like it could make a lot of tests a lot shorter!

I have shortened a few, hopefully I have not missed others that I could have shortened, I will have another look just to make sure.

Copy link
Copy Markdown
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

service = new ExpressionScriptEngine();
lookup = new SearchLookup(mapperService, (ignored, lookup) -> fieldData);
lookup = new SearchLookup(field -> field.equals("field") || field.equals("alias") ? fieldType : null,
(ignored, lookup) -> fieldData);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder what these were testing before. You change doesn't make it worse though!

I wonder if this change in general could make field aliases simpler. Like, in a follow up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered the same, especially why expression tests need to specifically test field aliases, that should not be necessary?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing that aliases at some point were different beasts and the test tested something. Or maybe we ended up copy and pasting stuff?

@javanna
Copy link
Copy Markdown
Contributor Author

javanna commented Oct 22, 2020

run elasticsearch-ci/default-distro

if (this.lookup == null) {
this.lookup = new SearchLookup(
mapperService,
this::getFieldType,
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this means the lookup does a tiny bit more than before: handling unmapped fields. I wonder if that is good or bad, but tests seem to be happy either way. If we want to preserve the previous behaviour, we can easily do so.

values.add(mapper.type());
}
if (TypeFieldType.NAME.equals(fieldType.name())) {
values = Collections.singletonList(((TypeFieldType)fieldType).getType());
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not fantastic, yet it does the job... do we have better options? I like that this way we still only need the lookup function as argument.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think we can actually remove this in master, as using the _type field in scripts is deprecated in 7.x. But you might want to keep it for the backport, and then remove it later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #64041

Copy link
Copy Markdown
Contributor

@romseygeek romseygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I particularly like all the lines that are removing Mockito imports, always a good sign!


DocLookup(MapperService mapperService, Function<MappedFieldType, IndexFieldData<?>> fieldDataLookup) {
this.mapperService = mapperService;
DocLookup(Function<String, MappedFieldType> fieldTypeLookup, Function<MappedFieldType, IndexFieldData<?>> fieldDataLookup) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could go even further in a follow up, and just have one String->IndexFieldData lookup?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting idea, and this could be applied to other places too, where all we do is look up the type by name and then look up fielddata by field type. I will look into this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into this and I found logic around the field type in the callers, so I will leave this for another time :)

values.add(mapper.type());
}
if (TypeFieldType.NAME.equals(fieldType.name())) {
values = Collections.singletonList(((TypeFieldType)fieldType).getType());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think we can actually remove this in master, as using the _type field in scripts is deprecated in 7.x. But you might want to keep it for the backport, and then remove it later.

@javanna javanna merged commit b1d55e2 into elastic:master Oct 22, 2020
javanna added a commit to javanna/elasticsearch that referenced this pull request Oct 22, 2020
…lastic#64017)

SearchLookup currently takes the whole MapperService as an argument, and it passes it over to DocLookup and FieldsLookup. In practice, what they all need is only a way to lookup field types given a field name.

With this commit, SearchLookup requires a Function<String,MappedFieldType> to be created rather than the whole MapperService. This helps clarifying the dependency between MapperService and SearchLookup, especially as we introduce runtime fields defined in the search request: they can be exposed at QueryShardContext level, through its lookup methods, rather than at the MapperService level which QueryShardContext depends on.
javanna added a commit that referenced this pull request Oct 26, 2020
…64017)

SearchLookup currently takes the whole MapperService as an argument, and it passes it over to DocLookup and FieldsLookup. In practice, what they all need is only a way to lookup field types given a field name.

With this commit, SearchLookup requires a Function<String,MappedFieldType> to be created rather than the whole MapperService. This helps clarifying the dependency between MapperService and SearchLookup, especially as we introduce runtime fields defined in the search request: they can be exposed at QueryShardContext level, through its lookup methods, rather than at the MapperService level which QueryShardContext depends on.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>refactoring :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team v7.11.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants