Skip to content

[CLAP] Fix logit scales dtype for fp16#25754

Merged
sanchit-gandhi merged 1 commit intohuggingface:mainfrom
sanchit-gandhi:clap-dtype
Aug 25, 2023
Merged

[CLAP] Fix logit scales dtype for fp16#25754
sanchit-gandhi merged 1 commit intohuggingface:mainfrom
sanchit-gandhi:clap-dtype

Conversation

@sanchit-gandhi
Copy link
Contributor

@sanchit-gandhi sanchit-gandhi commented Aug 25, 2023 •
edited
Loading

What does this PR do?

On some hardware, taking torch.log of a tensor in float16 on the CPU fails:

in __init__(self, config)
   1956         audio_config = config.audio_config
   1957 
-> 1958         self.logit_scale_a = nn.Parameter(torch.log(torch.tensor(config.logit_scale_init_value)))
   1959         self.logit_scale_t = nn.Parameter(torch.log(torch.tensor(config.logit_scale_init_value)))
   1960 

RuntimeError: "log_vml_cpu" not implemented for 'Half'

Note that this only failed for me on a Colab T4, but not on a Titan RTX (used to test #25682).

Let's take math.log then convert it to a tensor - this will respect the dtype of the model but not take torch.log of a float16 CPU param.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Aug 25, 2023 •
edited
Loading

The documentation is not available anymore as the PR was closed or merged.

ArthurZucker
ArthurZucker approved these changes Aug 25, 2023
View reviewed changes
Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On CPU you can't use half anyway no?

@sanchit-gandhi
Copy link
Contributor Author

sanchit-gandhi commented Aug 25, 2023 •
edited
Loading

Yep try with this:

import torch

torch.tensor(0).half().dtype()

Gives:

tensor(0., dtype=torch.float16)

Is used when we load diffusers pipelines in fp16 (load state dict in fp16 on cpu then move to cuda)

@sanchit-gandhi sanchit-gandhi merged commit 0770ce6 into huggingface:main Aug 25, 2023
@sanchit-gandhi sanchit-gandhi deleted the clap-dtype branch August 25, 2023 12:30
parambharat pushed a commit to parambharat/transformers that referenced this pull request Sep 26, 2023
@sanchit-gandhi @parambharat
[CLAP] Fix logit scales dtype for fp16 (huggingface#25754)
8bdeb54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ArthurZucker ArthurZucker ArthurZucker approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sanchit-gandhi @HuggingFaceDocBuilderDev @ArthurZucker