Prevent upcasting norm layers in prepare_model_for_kbit_training

[sergiopaniego]

What does this PR do?

This PR removes the upcasting of layers when preparing the model for kbit training.

Experiment with and without this below:

The results are the same and the memory consumption is reduced a bit when not upcasted.

Additionally, I'm experimenting errors when upcasting using GRPO + QLoRA + Qwen/Qwen2-7B-Instruct

    123 
    124     def forward(self, input: Tensor) -> Tensor:
--> 125         return F.linear(input, self.weight, self.bias)
    126 
    127     def extra_repr(self) -> str:

RuntimeError: expected scalar type Float but found Half

And removing this bit, the error doesn't happen

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

Who can review?

@qgallouedec @kashif @lewtun

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[kashif]

shall we also rename the model to reflect that its preperation for peft training only?

[qgallouedec]

thanks for the PR. After backtracking, this comes from https://github.com/artidoro/qlora/blob/7f4e95a68dc076bea9b3a413d2b512eca6d004e5/qlora.py#L400-L401. Is your experiment with QLoRA? Do you mind running the same experiment with Llama?

[sergiopaniego]

thanks for the PR. After backtracking, this comes from https://github.com/artidoro/qlora/blob/7f4e95a68dc076bea9b3a413d2b512eca6d004e5/qlora.py#L400-L401. Is your experiment with QLoRA? Do you mind running the same experiment with Llama?

My experiments are with QLoRa. I've tried with "meta-llama/Llama-3.2-3B-Instruct" and the same exception is raised without this change. With the change, it works (notebook for reference)

[qgallouedec]

I also looked in detail at Qwen and Llama, and tried to evaluate the potential impact in terms of numerical differences on both the forward and backward passes. In short, there’s no difference, unless the weights are really extreme. After analyzing all the RMSNorm layers of Qwen 3 and Llama 3.2, the weights are well within safe values and don’t appear problematic.

I also couldn’t find any online documentation suggesting or explaining that this upcasting would be important for any reason. Therefore, unless there’s evidence or information indicating a regression, I propose we merge this change.

Thank you for the work on this, @sergiopaniego!