Skip to content

Commit 64d57c3

Browse files
authored
[Model] [Config] Correctly identify granite-4.0-micro as non-hybrid model (#28563)
Signed-off-by: Thomas Parnell <[email protected]>
1 parent a1e7fa3 commit 64d57c3

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

vllm/config/model.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1619,6 +1619,13 @@ def is_attention_free(self) -> bool:
16191619

16201620
@property
16211621
def is_hybrid(self) -> bool:
1622+
# Handle granite-4.0-micro case which uses hybrid config but does not
1623+
# actually contain any non-attention layers.
1624+
layer_types = getattr(self.hf_config, "layer_types", None)
1625+
if layer_types is not None and all(
1626+
layer == "attention" for layer in layer_types
1627+
):
1628+
return False
16221629
return self._model_info.is_hybrid
16231630

16241631
@property

0 commit comments

Comments
 (0)