Skip to content

MB-61640: Fuzzy Dynamic Scoring#2056

Merged
Likith101 merged 2 commits intomasterfrom
fuzzyScoring
Nov 21, 2024
Merged

MB-61640: Fuzzy Dynamic Scoring#2056
Likith101 merged 2 commits intomasterfrom
fuzzyScoring

Conversation

@Likith101
Copy link
Member

@Likith101 Likith101 commented Jul 24, 2024

  • Added levenshtein distance calculation for fuzzy and wildcard searchers
  • Added new implementations of certain functions to allow passing of edit distances per term
  • Multiplied boosts by inverse of edit distance for score calculation

@CascadingRadium
Copy link
Member

also please add unit tests to show the effect (with and w/o boost) across match and match_phrase queries thanks

@abhinavdangeti abhinavdangeti modified the milestone: v2.4.3 Aug 6, 2024
@abhinavdangeti abhinavdangeti added this to the v2.5.0 milestone Sep 18, 2024
@Likith101 Likith101 changed the title MB-61640: Toy: Fuzzy and Wildcard dynamic scoring MB-61640: Fuzzy Dynamic Scoring Oct 22, 2024
Likith101 and others added 2 commits November 20, 2024 10:10
 - Added new implementations of certain functions to allow passing of edit distances per term
 - Propagated edit distances from vellum
 - Boosting each term based on the inverse of its fuzziness
@Likith101 Likith101 merged commit 38f1737 into master Nov 21, 2024
@abhinavdangeti abhinavdangeti deleted the fuzzyScoring branch November 22, 2024 15:24
project-mirrors-bot-tu bot pushed a commit to project-mirrors/forgejo-as-gitea-fork that referenced this pull request Apr 6, 2025
…-gitea#7468)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/blevesearch/bleve/v2](https://github.com/blevesearch/bleve) | require | minor | `v2.4.4` -> `v2.5.0` |

---

### Release Notes

<details>
<summary>blevesearch/bleve (github.com/blevesearch/bleve/v2)</summary>

### [`v2.5.0`](https://github.com/blevesearch/bleve/releases/tag/v2.5.0)

[Compare Source](blevesearch/bleve@v2.4.4...v2.5.0)

##### Bug Fixes

-   Exact hits to score higher than fuzzy hits, with blevesearch/bleve#2056
-   Fix boosting during hybrid search that involves text + nearest neighbor, with blevesearch/bleve#2127
-   Addressed bug in IP field handling while highlighting, with blevesearch/bleve#2142
-   Graceful error handling within registry, with blevesearch/bleve#2151
-   `http/` package (meant for demo purposes) removed from repository to remove vulnerability - [CVE-2022-31022](GHSA-9w9f-6mg8-jp7w), relocated to within https://github.com/blevesearch/bleve-explorer
-   Geo radius queries will now advertise distances (within sort values) in readable format, with blevesearch/bleve#2137

##### Improvements

-   Vector search requires `faiss` dynamic library to be built from [blevesearch/faiss@352484e](https://github.com/blevesearch/faiss/tree/352484e0fc9d1f8f46737841efe5f26e0f383f71) which is a modified version of [v1.10.0](https://github.com/facebookresearch/faiss/releases/tag/v1.10.0)
-   Support for **BM25 scoring**, see: [scoring.md](https://github.com/blevesearch/bleve/blob/v2.5.0/docs/scoring.md#bm25)
-   Support for **synonyms' search**, see: [synonyms.md](https://github.com/blevesearch/bleve/blob/v2.5.0/docs/synonyms.md)
-   **Significant performance improvements in pre-filtered vector search**, with blevesearch/bleve#2169 + dependent changes
-   `auto` fuzziness detection with blevesearch/bleve#2060
-   Ability to affect ingestion/drain rate by tuning persister workers with blevesearch/bleve#2100
-   Additional config in merge policy for improved merger behavior, with blevesearch/bleve#2134
-   Geo improvements: footprint reduction for polygons, better validation and graceful error handling, with blevesearch/bleve#2162 + blevesearch/bleve#2158 + blevesearch/bleve#2165
-   Upgrade to RoaringBitmap/roaring@v2.4.5, etcd.io/bbolt@v1.4.0
-   More metrics

##### Milestone

-   [v2.5.0](https://github.com/blevesearch/bleve/milestone/24)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "* 0-3 * * *" (UTC), Automerge - "* 0-3 * * *" (UTC).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yMjIuMSIsInVwZGF0ZWRJblZlciI6IjM5LjIyMi4xIiwidGFyZ2V0QnJhbmNoIjoiZm9yZ2VqbyIsImxhYmVscyI6WyJkZXBlbmRlbmN5LXVwZ3JhZGUiLCJ0ZXN0L25vdC1uZWVkZWQiXX0=-->

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7468
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Shiny Nematoda <snematoda@noreply.codeberg.org>
Co-authored-by: Renovate Bot <forgejo-renovate-action@forgejo.org>
Co-committed-by: Renovate Bot <forgejo-renovate-action@forgejo.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants