Skip to content

Add bfloat16 extension array#1002

Merged
wjones127 merged 12 commits intomainfrom
wjones127/bfloat16
Jul 14, 2023
Merged

Add bfloat16 extension array#1002
wjones127 merged 12 commits intomainfrom
wjones127/bfloat16

Conversation

@wjones127
Copy link
Copy Markdown
Contributor

Closes #994

from ml_dtypes import bfloat16
from lance.arrow import PandasBFloat16Array, BFloat16Array

np_array = np.array([1.1, 2.1, 3.4], dtype=bfloat16)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make sure that it works with tensorflow bf16?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tensorflow converts to numpy with the ml_dtypes.bfloat16, so I think we are good here 👍

@wjones127 wjones127 force-pushed the wjones127/bfloat16 branch 2 times, most recently from ff466a2 to ba01bb7 Compare July 14, 2023 21:43
@wjones127 wjones127 force-pushed the wjones127/bfloat16 branch from ba01bb7 to 521fc09 Compare July 14, 2023 21:47
@wjones127 wjones127 requested a review from eddyxu July 14, 2023 21:58
Copy link
Copy Markdown
Member

@eddyxu eddyxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

please make sure every file has a license header.

@wjones127 wjones127 merged commit 14046dc into main Jul 14, 2023
@wjones127 wjones127 deleted the wjones127/bfloat16 branch July 14, 2023 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support bf16 as extension type for a column

2 participants