Conversation
MekkCyber
left a comment
There was a problem hiding this comment.
LGTM thanks for adding this 🔥 ! can you please add some tests in tests/quantization/ggml/ ?
tests/quantization/ggml/test_ggml.py
Outdated
| try: | ||
| original_model = AutoModelForCausalLM.from_pretrained( | ||
| original_model_id, | ||
| torch_dtype=torch.float16, | ||
| trust_remote_code=True, | ||
| device_map="auto", | ||
| ) | ||
| except Exception as e: | ||
| self.skipTest(f"Original Deci model not available for comparison: {e}") | ||
| return |
There was a problem hiding this comment.
Why do we need this try: ... excetpt: .... statement in test? Should't the test failed if it encounter an Exception?
There was a problem hiding this comment.
Can you address this too?
|
hey , any update on this ? |
I think after resolving conflicts we're good to go? Looking forward to this pr! Thanks. |
Isotr0py
left a comment
There was a problem hiding this comment.
Sorry for missing this thread! LGTM
tests/quantization/ggml/test_ggml.py
Outdated
| try: | ||
| original_model = AutoModelForCausalLM.from_pretrained( | ||
| original_model_id, | ||
| torch_dtype=torch.float16, | ||
| trust_remote_code=True, | ||
| device_map="auto", | ||
| ) | ||
| except Exception as e: | ||
| self.skipTest(f"Original Deci model not available for comparison: {e}") | ||
| return |
There was a problem hiding this comment.
Can you address this too?
|
thanks for keeping up with the pr ❤️ |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: ggml |
What does this PR do?
GGUF support for deci
Fixes #37736
Before submitting
Who can review?
@MekkCyber