Skip to content

Explain Function Score Query 8.15 Backport#111864

Merged
john-wagster merged 4 commits intoelastic:8.15from
john-wagster:explain-function-score-query-815
Aug 15, 2024
Merged

Explain Function Score Query 8.15 Backport#111864
john-wagster merged 4 commits intoelastic:8.15from
john-wagster:explain-function-score-query-815

Conversation

@john-wagster
Copy link
Copy Markdown
Contributor

Addressing an NPE found when building a custom plugin leveraging a script.
#109177

This occurs when calling execute on a ScoreScript like this:

public double execute(ExplanationHolder explanation) {
    explanation.set("An example optional custom description to explain details for this script's execution; we'll provide a default one if you leave this out.");
   ...
}
Example Returned Hit with the Explanation Customized
{
  "_shard" : "[test][0]",
  "_node" : "6aOP4_Q5SOGIyr4oIxj0TQ",
  "_index" : "test",
  "_id" : "2",
  "_score" : 0.5685853,
  "_source" : {
    "important_field" : "foo foo foo"
  },
  "_explanation" : {
    "value" : 0.5685853,
    "description" : "function score, product of:",
    "details" : [
      ...
      {
        "value" : 3.0,
        "description" : "min of:",
        "details" : [
          {
            "value" : 3.0,
            "description" : "An example optional custom description to explain details for this script's execution; we'll provide a default one if you leave this out.",
            "details" : [
              {
                "value" : 0.18952844,
                "description" : "_score: ",
                "details" : [
                  {
                    "value" : 0.18952844,
                    "description" : "weight(important_field:foo in 1) [PerFieldSimilarity], result of:",
                    "details" : [
                      {
                        "value" : 0.18952844,
                        "description" : "score(freq=3.0), computed as boost * idf * tf from:",
                        "details" : [
                          {
                            "value" : 2.2,
                            "description" : "boost",
                            "details" : [ ]
                          },
                          {
                            "value" : 0.13353139,
                            "description" : "idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from:",
                            "details" : [
                              {
                                "value" : 3,
                                "description" : "n, number of documents containing term",
                                "details" : [ ]
                              },
                              {
                                "value" : 3,
                                "description" : "N, total number of documents with field",
                                "details" : [ ]
                              }
                            ]
                          },
                          {
                            "value" : 0.6451613,
                            "description" : "tf, computed as freq / (freq + k1 * (1 - b + b * dl / avgdl)) from:",
                            "details" : [
                              {
                                "value" : 3.0,
                                "description" : "freq, occurrences of term within document",
                                "details" : [ ]
                              },
                              {
                                "value" : 1.2,
                                "description" : "k1, term saturation parameter",
                                "details" : [ ]
                              },
                              {
                                "value" : 0.75,
                                "description" : "b, length normalization parameter",
                                "details" : [ ]
                              },
                              {
                                "value" : 3.0,
                                "description" : "dl, length of field",
                                "details" : [ ]
                              },
                              {
                                "value" : 2.0,
                                "description" : "avgdl, average length of field",
                                "details" : [ ]
                              }
                            ]
                          }
                        ]
                      }
                    ]
                  }
                ]
              }
            ]
          },
          {
            "value" : 3.4028235E38,
            "description" : "maxBoost",
            "details" : [ ]
          }
        ]
      }
    ]
  }
}
What the Hit Would Normally Look Like Without the Custom Explanation
{
  "_shard" : "[test][0]",
  "_node" : "Fg3ne_PXSi2ljhqljA6QOQ",
  "_index" : "test",
  "_id" : "2",
  "_score" : 0.5685853,
  "_source" : {
    "important_field" : "foo foo foo"
  },
  "_explanation" : {
    "value" : 0.5685853,
    "description" : "function score, product of:",
    "details" : [
      ...
      {
        "value" : 3.0,
        "description" : "min of:",
        "details" : [
          {
            "value" : 3.0,
            "description" : "script score function, computed with script:\"Script{type=inline, lang='expert_scripts', idOrCode='pure_df', options={}, params={field=important_field, term=foo}}\"",
            "details" : [
              {
                "value" : 0.18952844,
                "description" : "_score: ",
                "details" : [
                  {
                    "value" : 0.18952844,
                    "description" : "weight(important_field:foo in 1) [PerFieldSimilarity], result of:",
                    "details" : [
                      {
                        "value" : 0.18952844,
                        "description" : "score(freq=3.0), computed as boost * idf * tf from:",
                        "details" : [
                          {
                            "value" : 2.2,
                            "description" : "boost",
                            "details" : [ ]
                          },
                          {
                            "value" : 0.13353139,
                            "description" : "idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from:",
                            "details" : [
                              {
                                "value" : 3,
                                "description" : "n, number of documents containing term",
                                "details" : [ ]
                              },
                              {
                                "value" : 3,
                                "description" : "N, total number of documents with field",
                                "details" : [ ]
                              }
                            ]
                          },
                          {
                            "value" : 0.6451613,
                            "description" : "tf, computed as freq / (freq + k1 * (1 - b + b * dl / avgdl)) from:",
                            "details" : [
                              {
                                "value" : 3.0,
                                "description" : "freq, occurrences of term within document",
                                "details" : [ ]
                              },
                              {
                                "value" : 1.2,
                                "description" : "k1, term saturation parameter",
                                "details" : [ ]
                              },
                              {
                                "value" : 0.75,
                                "description" : "b, length normalization parameter",
                                "details" : [ ]
                              },
                              {
                                "value" : 3.0,
                                "description" : "dl, length of field",
                                "details" : [ ]
                              },
                              {
                                "value" : 2.0,
                                "description" : "avgdl, average length of field",
                                "details" : [ ]
                              }
                            ]
                          }
                        ]
                      }
                    ]
                  }
                ]
              }
            ]
          },
          {
            "value" : 3.4028235E38,
            "description" : "maxBoost",
            "details" : [ ]
          }
        ]
      }
    ]
  }
}

allowing for a custom explanation to be passed through as part of supporting building a plugin with a custom script score; previously threw an npe
@elasticsearchmachine elasticsearchmachine added v8.15.1 needs:triage Requires assignment of a team area label labels Aug 13, 2024
@john-wagster john-wagster added :Search Relevance/Search Catch all for Search Relevance >bug and removed needs:triage Requires assignment of a team area label labels Aug 13, 2024
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Aug 13, 2024
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Hi @john-wagster, I've created a changelog YAML for you.

@john-wagster john-wagster changed the title Explain function score query 815 Explain Function Score Query 8.15 Backport Aug 13, 2024
Copy link
Copy Markdown
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need reviews for backports unless something substantial changes :)


- requires:
cluster_features: [ "gte_v8.15.1" ]
reason: "bug fixed where explanations were throwing npe prior to 8.16"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
reason: "bug fixed where explanations were throwing npe prior to 8.16"
reason: "bug fixed where explanations were throwing npe prior to 8.15.1"

@john-wagster john-wagster merged commit 8ab481c into elastic:8.15 Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport >bug :Search Relevance/Search Catch all for Search Relevance Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v8.15.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants