Introducing `AutoPeftModelForxxx` by younesbelkada · Pull Request #694 · huggingface/peft

younesbelkada · 2023-07-13T11:43:41Z

This PR introduces a new paradigm, AutoPeftModelForxxx intended for users that want to rapidly load and run peft models.
Currently a user needs to run all these steps:

from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM

peft_config = PeftConfig.from_pretrained("ybelkada/opt-350m-lora") 
base_model_path = peft_config.base_model_name_or_path

transformers_model = AutoModelForCausalLM.from_pretrained(base_model_path, device_map="auto", load_in_8bit=True)
peft_model = PeftModel.from_pretrained(transformers_model, peft_config)

to load a peft model from the Hub or locally, whereas now they could just do:

from peft import AutoPeftModelForCausalLM

peft_model = AutoPeftModelForCausalLM.from_pretrained("ybelkada/opt-350m-lora")

cc @pacman100 @BenjaminBossan

TODOs:

add tests
add docs

HuggingFaceDocBuilderDev · 2023-07-13T11:48:57Z

The documentation is not available anymore as the PR was closed or merged.

BenjaminBossan

This looks quite good to me, I have only minor comments.

Admittedly, I'm not an expert in the concept of the Auto* models, so I can't comment on the overall design of this feature. Let's see what Sourab has to add to this.

src/peft/auto.py

BenjaminBossan · 2023-07-13T14:37:24Z

tests/test_auto.py

+)
+
+
+class PeftAutoModelTester(unittest.TestCase):


The individual tests all look very similar. I wonder if they could be parametrized.

Yeah they are quite similar however, to check if the models are effectively converted in bfloat16 for instance I check some custom module attributes for each case. Maybe let's keep that as it is

I see. Those could be split into separate tests or could be parametrized via operator.attrgetter but it's not super important.

BenjaminBossan · 2023-07-13T15:10:57Z

tests/test_auto.py

+)
+
+
+class PeftAutoModelTester(unittest.TestCase):


I see. Those could be split into separate tests or could be parametrized via operator.attrgetter but it's not super important.

BenjaminBossan · 2023-07-13T15:14:06Z

src/peft/auto.py

+        )
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path, *peft_model_args, **kwargs):


Ah, I just noticed this distinction between *args and **kwargs now. IMHO it's not super intuitive that it works that way. Not sure what would be a better solution, but it should be at least documented what *peft_model_args are.

As there are only 3 params adapter_name, adapter_name, config apart from the base model and peft model path that is already passed, and the fact that they are all the same across the supported tasks and are kwargs, would it make sense to just have them explicitly mentioned here too?

Thanks for the valuable feedback, yes that would makes totally sense

BenjaminBossan · 2023-07-13T15:15:15Z

src/peft/auto.py

+    _target_peft_class = None
+
+    def __init__(self, *args, **kwargs):
+        raise TypeError(


Oh, I see now that transformers uses EnvironmentError and that's why you used it. I still think it's the wrong error to raise, but maybe it should be used for consistency if there is some code somewhere that catches this error specifically??

I see that makes sense, will revert it to EnvironmentError for consistency with transformers!

Maybe add a comment that explains why.

Sure, just added a comment

pacman100

Great work @younesbelkada, I really like the simplified UX ✨. I have a few queries based on the offline discussion.

Methods such as LoRA, AdaLora, IA3, AdaptionPrompt etc are widely applicable to tasks other than those explicitly supported as they do inline changes to the modules, such as ASR, Image Captioning using BLIP, Stable DIffusion, etc. What are your thoughts on going about it as users might expect it to work for those too? Or do we restrict this API to only the explicitly supported NLP tasks?
Left a comment

pacman100 · 2023-07-14T03:49:02Z

src/peft/auto.py

+        )
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path, *peft_model_args, **kwargs):


As there are only 3 params adapter_name, adapter_name, config apart from the base model and peft model path that is already passed, and the fact that they are all the same across the supported tasks and are kwargs, would it make sense to just have them explicitly mentioned here too?

younesbelkada

Hi @pacman100
Thanks for your review, regarding your first point, we could probably do a first version with only officially supported NLP tasks and do a second iteration to add a new auto mapping class that group the model classes based on modalities

pacman100

Thank you @younesbelkada for iterating, LGTM!

* working v1 for LMs * added tests. * added documentation. * fixed ruff issues. * added `AutoPeftModelForFeatureExtraction` . * replace with `TypeError` * address last comments * added comment.

* update to `prepare_model_for_kbit_training` from deprecated `prepare_model_for_int8_training` and add `use_gradient_checkpointing=args.gradient_checkpointing` to automatically follow the gradient checkpointing choice is also the workaround for huggingface#694 * workaround for gradient checkpointing issue calling model.gradient_checkpointing_enable() twice causes issues this workaround calls it in prepare_model_for_kbit_training and then changes the arg to false to make sure it isn't called again in huggingface trainer inner loop also changes stack_llama_2 sft trainer to use correct device map for ddp training so that you can test this issue

working v1 for LMs

2ddc3f3

younesbelkada added 5 commits July 13, 2023 12:43

Merge remote-tracking branch 'upstream/main' into add-auto-peft-model

cde5a99

added tests.

2f43426

added documentation.

1b413bc

fixed ruff issues.

c69a372

added AutoPeftModelForFeatureExtraction .

24a72e9

younesbelkada marked this pull request as ready for review July 13, 2023 14:13

younesbelkada requested review from BenjaminBossan and pacman100 July 13, 2023 14:13

BenjaminBossan approved these changes Jul 13, 2023

View reviewed changes

replace with TypeError

60eef55

BenjaminBossan reviewed Jul 13, 2023

View reviewed changes

pacman100 reviewed Jul 14, 2023

View reviewed changes

address last comments

f1e5c98

younesbelkada commented Jul 14, 2023

View reviewed changes

younesbelkada requested a review from pacman100 July 14, 2023 07:20

added comment.

c1fab76

pacman100 approved these changes Jul 14, 2023

View reviewed changes

younesbelkada merged commit 0675541 into huggingface:main Jul 14, 2023

younesbelkada deleted the add-auto-peft-model branch July 14, 2023 09:07

This was referenced Jul 14, 2023

[Auto] Support AutoPeftModel for custom HF models #707

Merged

[core] PEFT integration huggingface/transformers#24827

Closed

Conversation

younesbelkada commented Jul 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Jul 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pacman100 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

younesbelkada left a comment

Choose a reason for hiding this comment

Uh oh!

pacman100 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

younesbelkada commented Jul 13, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Jul 13, 2023 •

edited

Loading

pacman100 left a comment •

edited

Loading