Add CogVLM#27718
Conversation
There was a problem hiding this comment.
cc @ydshieh for this model, I need to store 2 attributes to the processor, however we currently don't have a processor_config.json file. Can we add support for this in from_pretrained and save_pretrained?
There was a problem hiding this comment.
cc @ydshieh for this model, I need to store 2 attributes to the processor, however we currently don't have a processor_config.json file. Can we add support for this in from_pretrained and save_pretrained?
fc83063 to
d3c5fc3
Compare
|
A cleaner implementation I'm working on is here: https://github.com/NielsRogge/transformers/tree/add_cogvlm_cleaner. It implements the model like llava, by adding the image tokens inside the model, rather than creating them in the processor class. |
|
Closing this one in favor of the PR above. |
What does this PR do?
This PR adds CogVLM natively into the Transformers library (it's already usable with
trust_remote_code=True, but with this PR one can run it without the xformers, einops and triton dependencies).To do:
FastRotaryEmbeddingoptional)Processor#27761