LoftQ initialisation support
I am loading a 4B parameter model with `load_in_4bit=True`. I am trying to use LoftQ initialisation with the following code:
```python
from unsloth import FastModel
from peft import LoftQConfig
model = FastModel.get_peft_model(
model,
r=128,
lora_alpha=90,
lora_dropout=0,
bias="none",
random_state=42,
use_rslora=True,
target_modules="all-linear",
use_gradient_checkpointing="unsloth",
weight_decay=0.01,
init_lora_weights="loftq",
loftq_config=LoftQConfig(loftq_bits=4, loftq_iter=1),
)
```
When I try to train with these settings I get the following error:
```bash
ValueError: Unsloth: You are using loftq init, yet load_in_4bit = True was set.
Reload your model without any quantization by setting load_in_4bit = False
```
In my understanding, LoftQ is beneficial for fine-tuning quantized models. If I am not allowed to load in the 4-bit quantized model, then what is the point of LoftQ initialisation?
I was unable to find any relevant documentation regarding this. Thus, it be great if anyone can provide an explanation of why this is the case when using LoftQ with Unsloth.
My (relevant) package versions are the following:
```
"bitsandbytes>=0.49.2",
"peft>=0.19.1",
"trl>=0.23.0",
"unsloth>=2025.11.1",
```
0 条评论