- latest release: 2025.12.6
- local - Jupyter Notebook
- not relevant
4.https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(0_6B)-Phone_Deployment.ipynb
- unsloth 2025.12.6
- not relevant
Description:
Running the:
from unsloth import FastLanguageModel
import torch
# Models supported for Phone Deployment
fourbit_models = [
"unsloth/Qwen3-4B", # Any Qwen3 model like 0.6B, 4B, 8B, 32B
"unsloth/Qwen3-32B",
"unsloth/Llama-3.1-8B-Instruct", # Llama 3 models work
"unsloth/Llama-3.3-70B-Instruct",
"unsloth/gemma-3-270m-it", # Gemma 3 models work
"unsloth/gemma-3-27b-it",
"unsloth/Qwen2.5-7B-Instruct", # And more models!
"unsloth/Phi-4-mini-instruct",
] # More models at https://huggingface.co/unsloth
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/Qwen3-0.6B",
max_seq_length = 1024,
full_finetuning = True,
qat_scheme = "phone-deployment", # Flag for phone deployment
)prevents next cells from running, as it results in an error:
[IPKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
File "/opt/conda/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell
await result
File "/opt/conda/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 366, in execute_request
await super().execute_request(stream, ident, parent)
File "/opt/conda/lib/python3.11/site-packages/ipykernel/kernelbase.py", line 827, in execute_request
reply_content = await reply_content
^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 458, in do_execute
res = shell.run_cell(
^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/ipykernel/zmqshell.py", line 663, in run_cell
return super().run_cell(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3114, in run_cell
with self._tee(channel="stdout"), self._tee(channel="stderr"):
File "/opt/conda/lib/python3.11/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3073, in _tee
stream.write = write
^^^^^^^^^^^^
AttributeError: 'HidePrintMessage' object attribute 'write' is read-onlyThe problem
in unsloth/import_fixes.py
if os.environ.get("UNSLOTH_ENABLE_LOGGING", "0") != "1":
import sys
# Apply to stderr for FBGEMM
sys.stderr = HidePrintMessage(sys.stderr) # <<--- This line causes the error and prevents all other cells from running.Work around for me
Comment the sys.stderr assignment:
if os.environ.get("UNSLOTH_ENABLE_LOGGING", "0") != "1":
import sys
# Apply to stderr for FBGEMM
#sys.stderr = HidePrintMessage(sys.stderr)
4.https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(0_6B)-Phone_Deployment.ipynb
Description:
Running the:
prevents next cells from running, as it results in an error:
The problem
in unsloth/import_fixes.py
Work around for me
Comment the sys.stderr assignment: