How to load float16 or int8 llava1.6 model #247

Closed

Labels

opened

on Feb 29, 2024

Is there a way to use the original llava1.6 model (bf16) as fp16 or int8 dtype?

Metadata

Assignees

No one assigned

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests