Skip to content

Fix f16 to f32 coercion to meet Trino f32 minimum#64

Merged
jackye1995 merged 1 commit into
lance-format:mainfrom
jtuglu1:support-variable-bit-width-vectors
Mar 10, 2026
Merged

Fix f16 to f32 coercion to meet Trino f32 minimum#64
jackye1995 merged 1 commit into
lance-format:mainfrom
jtuglu1:support-variable-bit-width-vectors

Conversation

@jtuglu1

@jtuglu1 jtuglu1 commented Mar 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Changes

Fixes issue where f16 columns were incorrectly being casted to f32. Since Trino does not support float values with width < 32bits, this widens f16 columns to f32 when doing reads to prevent this issue.

Stacktrace:

Caused by: java.lang.ClassCastException: class org.apache.arrow.vector.Float2Vector cannot be cast to class org.apache.arrow.vector.Float4Vector (org.apache.arrow.vector.Float2Vector and org.apache.arrow.vector.Float4Vector are in unnamed module of loader io.trino.server.PluginClassLoader @24e3d69f)
	at io.trino.plugin.lance.LanceArrowToPageScanner.lambda$convertType$6(LanceArrowToPageScanner.java:337)
	at io.trino.plugin.lance.LanceArrowToPageScanner.writeVectorValues(LanceArrowToPageScanner.java:425)
	at io.trino.plugin.lance.LanceArrowToPageScanner.convertType(LanceArrowToPageScanner.java:337)
	... 45 more

@jtuglu1 jtuglu1 force-pushed the support-variable-bit-width-vectors branch from 31bd519 to 542eebf Compare March 10, 2026 03:02
@jtuglu1 jtuglu1 marked this pull request as ready for review March 10, 2026 03:02
@jtuglu1 jtuglu1 requested a review from jackye1995 March 10, 2026 03:02
@jtuglu1 jtuglu1 force-pushed the support-variable-bit-width-vectors branch 3 times, most recently from 2c5e755 to d44951b Compare March 10, 2026 03:54
@jtuglu1 jtuglu1 force-pushed the support-variable-bit-width-vectors branch from d44951b to 76cabf6 Compare March 10, 2026 06:01
jackye1995
jackye1995 approved these changes Mar 10, 2026
View reviewed changes

@jackye1995 jackye1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks for the fix!

@jackye1995 jackye1995 merged commit 4e610a2 into lance-format:main Mar 10, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jackye1995 jackye1995 jackye1995 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jtuglu1 @jackye1995