Skip to content

Allow rescorer with field collapsing#107779

Merged
jimczi merged 12 commits intoelastic:mainfrom
jimczi:collapse_with_rescore
Apr 29, 2024
Merged

Allow rescorer with field collapsing#107779
jimczi merged 12 commits intoelastic:mainfrom
jimczi:collapse_with_rescore

Conversation

@jimczi
Copy link
Copy Markdown
Contributor

@jimczi jimczi commented Apr 23, 2024

This change adds the support for rescoring collapsed documents.
The rescoring is applied on the top document per group on each shard.

Closes #27243

jimczi added 2 commits April 23, 2024 16:27
@jimczi
Allow rescorer with field collapsing
66f4754 
This change adds the support for rescoring collapsed documents.
The rescoring is applied on the top document per group on each shard.

Closes elastic#27243
@jimczi
fix compileTest
4fa5392
@jimczi jimczi added >feature :Search/Search Search-related issues that do not fall into other categories v8.15.0 labels Apr 23, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 23, 2024

Documentation preview:

@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Apr 23, 2024
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented Apr 23, 2024

Pinging @elastic/es-search (Team:Search)

@jimczi jimczi requested a review from a team as a code owner April 23, 2024 16:46
benwtrent
benwtrent reviewed Apr 23, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very exciting!

benwtrent
benwtrent approved these changes Apr 24, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic.

Do we think we could also add rescore support for inner hits in the future?

I realize inner-hits are optional for collapse, so we can discuss this later. But it would be good to provide rescoring for inner hits so that their scores & order reflect their collapsed fields. This may require an API change for the inner-hits configuration to accept a "rescore" parameter or something. I honestly don't know.

@jimczi
Copy link
Copy Markdown
Contributor Author

jimczi commented Apr 24, 2024

Do we think we could also add rescore support for inner hits in the future

That's a good question. Adding the rescoring for inner_hits should be trivial since we can copy the rescorer into the expand query. I wanted to add that initially but then realise that users may want to apply a different window_size when rescoring inside each group. The original window_size might be too big to apply on every group since that would be applied on each top docs request per group. I left it for a follow up and added a warning in the doc for now.

kderusso
kderusso approved these changes Apr 24, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@kderusso kderusso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for taking care of this!

@jimczi jimczi merged commit 4380cd1 into elastic:main Apr 29, 2024
@jimczi jimczi deleted the collapse_with_rescore branch April 29, 2024 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mark-vieira mark-vieira mark-vieira approved these changes

@benwtrent benwtrent benwtrent approved these changes

@kderusso kderusso kderusso approved these changes

Assignees

No one assigned

Labels

>feature :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team v8.15.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

support field collapsing + rescore

5 participants

@jimczi @elasticsearchmachine @mark-vieira @benwtrent @kderusso