[`core`] PEFT integration by younesbelkada · Pull Request #24827 · huggingface/transformers

younesbelkada · 2023-07-14T17:06:56Z

What does this PR do?

This PR is an attempt to tightly integrate PEFT library with transformers, by offering users the ability to load PEFT models out of the box from AutoModelForxxx.from_pretrained() if the local directory or the Hub model id contains adapter weights and adapter config.

import tempfile
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import AutoPeftModelForCausalLM

peft_model_id = "ybelkada/opt-350m-lora"
model = AutoModelForCausalLM.from_pretrained(peft_model_id)

with tempfile.TemporaryDirectory() as tmpdirname:
    peft_model = AutoPeftModelForCausalLM.from_pretrained(peft_model_id)
    peft_model.save_pretrained(tmpdirname)

    model = AutoModelForCausalLM.from_pretrained(tmpdirname)
    print(model)

Although this is similar to what have been introduced in huggingface/peft#694 this PR offers a direct integration with transformers

TODOs:

handle PeftModel.from_pretrained(xxx) kwargs
tests
docs (with the help of @stevhliu )

cc @sgugger @pacman100 @BenjaminBossan

younesbelkada · 2023-07-14T17:07:55Z

I would like to have first review of the draft if possible, to see if we are inline with the approach 🙏 @sgugger - Thanks !

sgugger

Thanks for working on this, left some initial comments!

src/transformers/modeling_utils.py

sgugger · 2023-07-14T17:17:24Z

src/transformers/modeling_utils.py

+
+        is_adapter_file = False
+        if is_peft_available():
+            is_adapter_file = peft_adapter_model_id is not None


Maybe raise an error if someone wants to load a model with an adapter file and peft is not installed? I don't know how common that case is, but it seems to me that the user would want to use the adapter file if it's there no?

I think this is a great idea, and I am keen to add that condition as well. However, I am worried about something, this means we'll need to look for the adapter_config.json file on the local dir or remote directory in any case and I want to avoid that and restrict only to users that have PEFT installed. What do you think? If you think that this is fine, happy to add it.

src/transformers/models/auto/auto_factory.py

HuggingFaceDocBuilderDev · 2023-07-14T17:25:46Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

pacman100

Thank you @younesbelkada for integration of PEFT deeply into transformers 🚀! A concern that I have now is that there are various ways of loading PEFT models now, i.e., PeftModel.from_pretrained(model, peft_model_id), AutoPeftModelForCausalLM.from_pretrained(peft_model_id) and now AutoModelForCausalLM.from_pretrained(peft_model_id); this may lead to confusion among end-users. We need to clearly document the preferred/recommended way. And this needs to be updated on the inference widgets too. WDTY?

younesbelkada · 2023-07-17T08:22:04Z

Thank you for your review @pacman100 !
I think the canonical way to load PEFT models for inference would still be to use PEFT classes (i.e. either AutoPeftModelForCausalLM or PeftModel - btw we should encourage users to use more and more AutoPeftModelForCausalLM instead of PeftModel)
This PR is intended to make things even easier for users and for further integrations with HF ecosystem (pipeline, diffusers) and it will be clearly documented. I also think we should update the inference widgets after the PEFT release.

sgugger

Left a couple more comments!

src/transformers/modeling_utils.py

src/transformers/models/auto/auto_factory.py

sgugger · 2023-07-17T10:56:07Z

tests/peft_integration/test_peft_models.py

+
+@require_peft
+@require_torch
+@slow


Can we also add one non-slow test with a small fake model?

Should be done, also added a new job to run peft tests thanks to @ydshieh

src/transformers/modeling_utils.py

[docs] PEFT + Transformer docs

sgugger

Mmm that's not the correct way to add a new peft job as it will always run on any PR, even if it shouldn't be run.

ydshieh · 2023-07-19T06:29:47Z

Mmm that's not the correct way to add a new peft job as it will always run on any PR, even if it shouldn't be run.

@younesbelkada

You can take a look

transformers/utils/tests_fetcher.py

Line 720 in 476be08

    
           examples_tests_to_run = [f for f in test_files_to_run if f.startswith("examples")]

(check examples_test_list.txt and examples_tests_to_run in the same file and the 2 CircleCI config files)
(you have to check against peft_integration in your case)

patrickvonplaten · 2023-07-19T08:46:18Z

The design as is would make it hard for diffusers to leverage transformes to load PEFT weights for transformers models.
In diffusers we have the following workflow:

from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
# now we have a loaded CLIPTextModel under `pipe.text_encoder`

pipe.load_lora(...)
# doing this means we would want to call `text_encoder.load_adapter(...)` under the hood

This means we necessarily need transformers to support the ability to load lora weights into already instantiated models, just like MMS allows it via load_adapter - see: https://huggingface.co/docs/transformers/v4.31.0/en/model_doc/wav2vec2#transformers.Wav2Vec2ForCTC.load_adapter.example

[Edit] We could come around it, by wrapping pipe.text_encoder into a PeftModel under the hood when doing pipe.load_lora(...) to transform CLIPTextModel into PeftModel but that:

would break some internal diffusers code
force us to wrap logic around transformers, e.g. we could just do pipe.text_encoder.load_adapter(...), but would have to first wrap every transformers model into a PEFT model

patrickvonplaten · 2023-07-19T09:31:12Z

From purely a transformers point of view, I would also struggle a bit with the following:

1.) PEFT weights seemingly should only be loaded with AutoModel, which is restrictive as there is no need to go over the AutoModel class if one knows the model.

It does look like the following would be possible:

from transformers import LlamaForCausalLM

model =  LlamaForCausalLM.from_pretrained("tloen/alpaca-lora-7b")

but then model.__class__ is of type PeftModel which would be confusing to me - I used a class method of LlamaForCausalLM

2.) I don't like that .from_pretrained(<peft/model/id>) more or less fully dispatches to the peft library instead of staying in Transformers' land. I imagined peft to be used as a utility library, not Transformers to dispatch to peft.

=> Could we not create a PeftModelMixin class so that peft operates more under the hood?

younesbelkada · 2023-08-22T17:10:57Z

Closing as #25077 got merged

younesbelkada added 2 commits July 14, 2023 17:01

working v1

d9bebbc

left a todo

80ce927

sgugger reviewed Jul 14, 2023

View reviewed changes

younesbelkada added 2 commits July 14, 2023 17:26

fix nits

6385a94

address comments with respect to cached file and auto_factory

032077a

younesbelkada mentioned this pull request Jul 15, 2023

[core] Better hub kwargs management huggingface/peft#712

Merged

younesbelkada added 7 commits July 15, 2023 19:40

style and added PeftModel.from_pretrained kwargs

0d76b5e

deal with non-auto mapping case.

2c4a88a

added test and PEFT library on Dockerfile.

90fead2

quality

1af8b1c

Merge remote-tracking branch 'upstream/main' into peft-integration

ebfacb0

added draft documentation - left some TODOs.

acc7e6d

fixed typo.

3a7d999

pacman100 reviewed Jul 17, 2023

View reviewed changes

younesbelkada requested a review from sgugger July 17, 2023 09:37

sgugger reviewed Jul 17, 2023

View reviewed changes

Update peft.md

911389a

stevhliu reviewed Jul 17, 2023

View reviewed changes

src/transformers/modeling_utils.py Show resolved Hide resolved

younesbelkada and others added 9 commits July 17, 2023 21:33

Merge pull request #3 from younesbelkada/peft-transformer

017dd75

[docs] PEFT + Transformer docs

addressed most of comments.

cd22c54

more nits

ef26bd5

added docstring on new kwargs.

ed8e388

add tiny random model

e1e21e0

add peft to CI

ab73ad0

add correct file

305d2c2

fix tests that were not ran

ef2f2ae

oops

f5c94e3

stevhliu and others added 2 commits July 18, 2023 10:26

add trainer integration/4bit and 8bit docs

694238d

fix nits.

74381da

sgugger reviewed Jul 18, 2023

View reviewed changes

Narsil mentioned this pull request Jul 21, 2023

Avoid importing all models when instantiating a pipeline #24960

Merged

younesbelkada mentioned this pull request Jul 25, 2023

[PEFT] Peft integration alternative design #25077

Merged

12 tasks

huggingface deleted a comment from github-actions bot Aug 16, 2023

younesbelkada closed this Aug 22, 2023

Conversation

younesbelkada commented Jul 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

TODOs:

Uh oh!

younesbelkada commented Jul 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sgugger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sgugger Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

younesbelkada Jul 15, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Jul 14, 2023

Uh oh!

pacman100 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

younesbelkada commented Jul 17, 2023

Uh oh!

sgugger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sgugger Jul 17, 2023

Choose a reason for hiding this comment

Uh oh!

younesbelkada Jul 18, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sgugger left a comment

Choose a reason for hiding this comment

Uh oh!

ydshieh commented Jul 19, 2023

Uh oh!

patrickvonplaten commented Jul 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

patrickvonplaten commented Jul 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

younesbelkada commented Aug 22, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

younesbelkada commented Jul 14, 2023 •

edited

Loading

younesbelkada commented Jul 14, 2023 •

edited

Loading

pacman100 left a comment •

edited

Loading

patrickvonplaten commented Jul 19, 2023 •

edited

Loading

patrickvonplaten commented Jul 19, 2023 •

edited

Loading