Is it possible to obtain separate features for multiple adapters in one inference? #2392
-
|
When loading multiple adapters, can the features of different adapters be obtained in one inference? Rather than the merged features of each adapter. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 2 replies
-
|
Could you clarify what exactly you want, maybe with some pseudo code? If you mean, for instance, having a model with multiple LoRA adapters and then calling |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your reply. I will now describe my problem in more detail. If I have a base model M loaded with two adapters, a and b, is it possible to obtain two outputs in a single inference—one generated by M + a and the other by M + b? This seems to align with what you described. |
Beta Was this translation helpful? Give feedback.
-
|
No, this is not possible. You should activate one adapter, call inference on it, then the other and call inference on that one. (In case you're wondering, if we built this feature into PEFT, it would work the same way under the hood, there would be no performance advantage) |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for your answer. Upon further consideration, I realize there were some issues with my previous thoughts, and you are right. |
Beta Was this translation helpful? Give feedback.
No, this is not possible. You should activate one adapter, call inference on it, then the other and call inference on that one. (In case you're wondering, if we built this feature into PEFT, it would work the same way under the hood, there would be no performance advantage)