[tests] Stricter generate + compilation test -- no recompilations allowed#37629
Conversation
|
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the |
| else: | ||
| # the 4D causal mask exists, it should be present in the base model (XXXModel class) or in its decoder. | ||
| base_model = getattr(self, self.base_model_prefix, self) | ||
| decoder = base_model.get_decoder() if hasattr(base_model, "get_decoder") else None |
There was a problem hiding this comment.
this fixes the compilation test for opt and helps with decoder-only whisper
|
cc @manueldeprada (since you mentioned you are interested in torch.compile) |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
zucchini-nlp
left a comment
There was a problem hiding this comment.
Thanks for adding the test! So this means all VLMs are actually compiling without any breaks, right. I am still wondering about the slow downs we experienced when compiling certain models
One q about the test: it guards torch>=2.6, are the runner installing the latest version? Would be nice to get this test running on each PR to avoid accidentally breaking gemma again
tests/generation/test_utils.py
Outdated
| # TODO (joao, raushan): do we need a custom `generate` in these models? can we call `super().generate`, as | ||
| # opposed to the inner model's `generate`? If yes, we would get a more standardized codebase | ||
| if "blip" in model.__class__.__name__.lower(): |
There was a problem hiding this comment.
No, blip cannot be fixed anymore, we risk to break it for existing hub repos 😢 I had a branch somewhere, but it didn't work
And it's not worth fixing, we can even skip compile tests for blip if it's making our life harder
There was a problem hiding this comment.
haha okay I'll remove the TODO 😢 (I'm a bit sad on the inside)
Yes, they are compilable without graph breaks. That by itself is useful, it means they can probably be exported with
Yes, they are running torch 2.6 (the runners' images install the latest version e.g. here). On this PR's |
zucchini-nlp
left a comment
There was a problem hiding this comment.
Great, happy to hear CI actually runs tests!
…owed (huggingface#37629) * tmp commit * stricter compilation test * trigger tests * rm todo
What does this PR do?
Follow-up to #37447
This PR upgrades
test_generate_compile_model_forwardto catch recompilation issues. This is done by a) activating recompilation logs b) catching recompilation messages in the logs. The improved test would have failed with the changes that broke gemma 2/3 + compile 🤗In the process, a few extra things were standardized in the test, and a few skips were removed.