Skip to content

[AutoGPTQ] Add correct installation of GPTQ library + fix slow tests#25713

Merged
younesbelkada merged 2 commits intohuggingface:mainfrom
younesbelkada:add-auto-gptq-dockerfile
Aug 24, 2023
Merged

[AutoGPTQ] Add correct installation of GPTQ library + fix slow tests#25713
younesbelkada merged 2 commits intohuggingface:mainfrom
younesbelkada:add-auto-gptq-dockerfile

Conversation

@younesbelkada
Copy link
Contributor

@younesbelkada younesbelkada commented Aug 24, 2023 •
edited
Loading

What does this PR do?

Per the instructions of installing auto-gptq library we need to slightly update the Dockerfile otherwise it will install the 0.3.2 version which is not compatible with the integration

Will also update some expected values to make the slow tests pass

cc @ydshieh @SunMarc

Can confirm the Docker image is built successfully

ydshieh
ydshieh approved these changes Aug 24, 2023
View reviewed changes
Copy link
Collaborator

@ydshieh ydshieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works for me. Thank you for confirming the build is successful.

So basically we want to install the same version but with cu118 support, right?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Aug 24, 2023 •
edited
Loading

The documentation is not available anymore as the PR was closed or merged.

@younesbelkada younesbelkada marked this pull request as ready for review August 24, 2023 10:11
@younesbelkada
Copy link
Contributor Author

younesbelkada commented Aug 24, 2023

Can confirm the slow tests now pass in the docker image

So basically we want to install the same version but with cu118 support, right?

It is slightly trickier than that, auto-gptq listed in pypi does not contain the latest supported versions (>= 0.4.1), one needs to install that directly through the command that I have shared. Also that way you get the pre-built wheels instead of building auto-gptq at each install (which is the case right now if you do pip install auto-gptq)

@younesbelkada younesbelkada requested a review from SunMarc August 24, 2023 10:13
SunMarc
SunMarc approved these changes Aug 24, 2023
View reviewed changes
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! Thanks for checking that the image is built correctly :)

@younesbelkada younesbelkada merged commit 584eeb5 into huggingface:main Aug 24, 2023
@younesbelkada younesbelkada deleted the add-auto-gptq-dockerfile branch August 24, 2023 12:57
parambharat pushed a commit to parambharat/transformers that referenced this pull request Sep 26, 2023
@younesbelkada @parambharat
[AutoGPTQ] Add correct installation of GPTQ library + fix slow tests (
234ddc2 
huggingface#25713)

* add correct installation of GPTQ library

* update tests values
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ydshieh ydshieh ydshieh approved these changes

@SunMarc SunMarc SunMarc approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@younesbelkada @HuggingFaceDocBuilderDev @ydshieh @SunMarc