Skip to content

Change internal representation of suggesters #16873

Merged
abeyad merged 6 commits intoelastic:feature-suggest-refactoringfrom
abeyad:suggester-wiring-refactoring
Mar 11, 2016
Merged

Change internal representation of suggesters #16873
abeyad merged 6 commits intoelastic:feature-suggest-refactoringfrom
abeyad:suggester-wiring-refactoring

Conversation

@abeyad
Copy link
Copy Markdown

@abeyad abeyad commented Feb 29, 2016

Change internal representation of suggesters to instances of SuggestBuilder instead of raw bytes.

@abeyad abeyad changed the title Change internal representation of suggesters WIP: Change internal representation of suggesters Feb 29, 2016
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to add this test to the existing SuggestBuilderTests and not create a new test class for this. If I understand the existing test in CompletionSuggestSearchIT correctly this is only testing pasting, so it should work, or am I missung something?

@cbuescher
Copy link
Copy Markdown
Member

@abeyad I only had a brief look, left a few minor comments. I like passing the suggesters to the parsing methods as additional argument like in the aggregations parsers. Maybe in a future step we might be able to collaps some of those. I didn't have a close look at the SearchSourceBuilder tests, but would also add a bit more randomization to the suggester setup in createSearchSourceBuilder() (maybe add a few more random suggestions, re-use the randomization from the individual tests if that is already possible?)

Also @areek might want to have another look.

@abeyad abeyad force-pushed the suggester-wiring-refactoring branch 2 times, most recently from aec2614 to da6004d Compare March 4, 2016 15:32
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@areek @cbuescher One thing I request you both to comment on in particular is adding the NamedWriteable parameter to the registerSuggester method. I don't know if this will cause any issues with registering custom suggester plugins, but adding this parameter is required so that we can register the associated builder with the custom suggester in the namedWriteableRegistry.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a getBuilderPrototype() method each suggester needs to provide which should return the appropriate builder prototype for each suggester. You could register the Suggester itself and then get its class and its builder prototype.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great idea, I made the change, thanks @cbuescher !

@abeyad abeyad force-pushed the suggester-wiring-refactoring branch from da6004d to 5e768cd Compare March 4, 2016 17:59
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think registering the smoothing models should also go somewhere into SearchModule. Since they are only used in PhraseSuggestionBuilder, I would probably put them alongside the build-in suggestion builders in configureSuggesters()

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@cbuescher
Copy link
Copy Markdown
Member

@abeyad I did a round of reviews, left some suggestions. I think you can remove the "WIP" from the title, looks already quiet far to me. After adressing some of the comments I think this only needs to wait for the CompletionSuggestion refactoring and then maybe a final review once all tests pass.

@abeyad abeyad changed the title WIP: Change internal representation of suggesters Change internal representation of suggesters Mar 7, 2016
@cbuescher
Copy link
Copy Markdown
Member

fyi I locally put an @AwaitsFix on CompletionSuggestSearchIT and ContextCompletionSuggestSearchIT to see if any later tests also break and found that in distribution/integ-test-zip there are several rest tests failing with "Suggestion must have name". Maybe we were to eager in enforcing this, maybe we only need to adapt the tests.

@abeyad
Copy link
Copy Markdown
Author

abeyad commented Mar 7, 2016

@cbuescher I think the tests you are referring to (the RestIT ones), those seem to fail with regards to the completion suggester only, no?

@cbuescher
Copy link
Copy Markdown
Member

Not sure, to me it looks more general, but maybe you are right. Just wanted to point it out.

@abeyad
Copy link
Copy Markdown
Author

abeyad commented Mar 7, 2016

Its possible, I started looking at it and showed some of the failures to @areek who thought they were related to the completion suggester only. I figured once that's in, we can iron out any remaining test failures further, because there is also the SuggestBuilderTests which only retrieve a random Term or Phrase SuggestionBuilder now, with the Completion commented out. So we probably still have some work to do with respect to the tests regardless.

@abeyad abeyad force-pushed the suggester-wiring-refactoring branch from 5e768cd to 05369e7 Compare March 7, 2016 16:57
@abeyad
Copy link
Copy Markdown
Author

abeyad commented Mar 7, 2016

I pushed my latest changes based on @cbuescher comments

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you could move the stuff from provideNamedWritableRegistry() here and then in provideNamedWritableRegistry() return the internal NamedWritableRegistry. Maybe less error prone when maintaining the test later to only have one registry around.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@cbuescher
Copy link
Copy Markdown
Member

@abeyad great, left two more minor comments.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just call it suggest?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@abeyad abeyad force-pushed the suggester-wiring-refactoring branch from 05369e7 to ec4af5e Compare March 7, 2016 21:16
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbuescher @areek I added this fromStart method parameter to differentiate, when parsing from x-content, between the situation where the suggest is part of a larger search request, and when the suggest is on its own as part of the suggest API. This may go away later when we merge the implementation of the search with suggest and standalone suggest into one, as @areek suggested before, but what I noticed was in some places, the parsing was failing and always returning back "suggestion name not found", even though there may have been other issues with the parsing. Also, valid JSON was not being parsed correctly by the suggest API. The reason is because in the search with suggest scenario, it has already parsed the starting token for the suggest object, whereas in the suggest API scenario, the starting token has not yet been parsed. In some places, we were adding another parser.nextToken() to account for this, but it wasn't being applied consistently and it was error prone to manage that. I feel having the method itself have a parameter that declares the intentions of the parsing (whether we have already parsed the starting { or not) is more clear and less error prone, but I welcome your thoughts.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the problem but I find adding this extra "control" parameter confusing. I would either make sure that the parser is always on the start token on the caller side (I think in SearchSourceBuilder this is always the case, in tests and in RestSuggestAction we should make sure it is) and otherwise throw an error, or check if parser.currentToken == null inside the SuggestBuilder parse method and advance the token pointer if so. This btw. is what ObjectParser#parse() does by default. I think this would be the best solution.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to this solution

@abeyad abeyad force-pushed the suggester-wiring-refactoring branch from 19c6e30 to 2f628b6 Compare March 10, 2016 05:09
@abeyad abeyad force-pushed the suggester-wiring-refactoring branch from 2f628b6 to a6662d7 Compare March 10, 2016 05:15
@abeyad
Copy link
Copy Markdown
Author

abeyad commented Mar 10, 2016

@cbuescher @areek I made some changes based on Areek's feedback.

@cbuescher
Copy link
Copy Markdown
Member

@abeyad looks good, unfortunately getting some compile errors bc. CompletionSuggesterBuilderTests.randomCompletionSuggestionBuilder() is not there. Also a few test are failing for me locally after I fix that.

@abeyad
Copy link
Copy Markdown
Author

abeyad commented Mar 10, 2016

Fixed and pushed changes.

On Thu, Mar 10, 2016 at 8:13 AM, Christoph Büscher <notifications@github.com

wrote:

@abeyad https://github.com/abeyad looks good, unfortunately getting
some compile errors bc.
CompletionSuggesterBuilderTests.randomCompletionSuggestionBuilder(), and
a few test are failing for me when I fix that.


Reply to this email directly or view it on GitHub
#16873 (comment)
.

@cbuescher
Copy link
Copy Markdown
Member

@abeyad thanks, still getting test errors (and later some compile issues in reindex module). I'm running gradle clean check in the meantime and fixing one by one, will let you know what I find (and maybe paste the diff somewhere). After that passes I think this PR should be merged with the feature branch.

@cbuescher
Copy link
Copy Markdown
Member

These are the tests (and minor compile thingies) that I found: https://gist.github.com/cbuescher/082435d20ddd279e3db4

@abeyad
Copy link
Copy Markdown
Author

abeyad commented Mar 11, 2016

LGTM

On Fri, Mar 11, 2016 at 9:52 AM, Christoph Büscher <notifications@github.com

wrote:

These are the tests (and minor compile thingies) that I found:
https://gist.github.com/cbuescher/082435d20ddd279e3db4


Reply to this email directly or view it on GitHub
#16873 (comment)
.

cbuescher and others added 2 commits March 11, 2016 17:58
@cbuescher
Copy link
Copy Markdown
Member

All tests pass now, LGTM, lets get this in.

abeyad pushed a commit that referenced this pull request Mar 11, 2016
Change internal representation of suggesters
@abeyad abeyad merged commit 31dcb3e into elastic:feature-suggest-refactoring Mar 11, 2016
@clintongormley clintongormley added :Search/Search Search-related issues that do not fall into other categories and removed :Search Refactoring labels Feb 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :Search/Search Search-related issues that do not fall into other categories v5.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants