fix: case when reranker returns invalid scores array #6777
Conversation
RomneyDa
left a comment
There was a problem hiding this comment.
@uinstinct see nitpick comment. Just to double check, has this been tested with voyage reranker?
core/llm/llms/Voyage.ts
Outdated
| data: Array<{ index: number; relevance_score: number }>; | ||
| }; | ||
|
|
||
| VoyageRerankSuccessResponseSchema.parse(data); |
There was a problem hiding this comment.
You can use the zod output here, it will be typesafe!
I'd recommend safeParse so you can gracefully handle invalid values
There was a problem hiding this comment.
actually I wanted the throw the error directly if parsing failed like we do here
continue/core/llm/llms/Voyage.ts
Lines 30 to 34 in b28d0f2
There was a problem hiding this comment.
@uinstinct the schema.parse will throw an error if parsing fails. i.e. if the status is 200 then presumably the parse with throw an error because the response won't match the expected format, which means the status check will never be reached
Yes had verified on Voyage reranker |
|
@uinstinct sorry for the confusion, EDIT I was wrong about the order here, the main concern now is that the parse will throw an ugly zod error so should be gracefully handled |
|
🎉 This PR is included in version 1.5.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 1.7.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Description
The following error used to happen when the reranker model returned an invalid response.
Failed to rerank retrieval results TypeError: Cannot read properties of undefined (reading 'sort'). More about the RCA in the ticket.scoresarray is valid in DocsContextProviderresolves CON-2661
Checklist
Screen recording or screenshot
[ When applicable, please include a short screen recording or screenshot - this makes it much easier for us as contributors to review and understand your changes. See this PR as a good example. ]
Tests
[ What tests were added or updated to ensure the changes work as expected? ]
Summary by cubic
Fixed a crash when the reranker model returned an invalid scores array by adding response validation and handling invalid cases.