Skip to content

Feature: Support Relu2 in FusedMoE fp8 cutlass path #27261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Feature: Support Relu2 in FusedMoE fp8 cutlass path #27261

wants to merge 2 commits into from
+51 −23

Conversation

@amirkl94
Copy link
Contributor

@amirkl94 amirkl94 commented Oct 21, 2025 •
edited by github-actions bot
Loading

Purpose

tomeras91
tomeras91 reviewed Oct 21, 2025
View reviewed changes
vllm/model_executor/layers/quantization/modelopt.py
apply_router_weight_on_input=apply_router_weight_on_input,
)
elif self.flashinfer_moe_backend == FlashinferMoeBackend.CUTLASS:
assert not renormalize
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. why did you remove the assert that renormalize is not True?
  2. Maybe worth asserting that activation is either "silu" or "relu2"?

vllm/model_executor/layers/fused_moe/flashinfer_cutlass_moe.py
expert_tokens_meta: mk.ExpertTokensMetadata | None,
apply_router_weight_on_input: bool | None,
):
assert activation == "silu", (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe worth asserting that activation is one of the keys in activation_str_to_value_map?

@amirkl94
Bugfix: Cutlass FP8 FusedMoE
90515e5 
When running cutlass FusedMoE FP8 the scaling factors that are passed are None.
This PR passes the correct scaling factors and enables the relevant test.

Signed-off-by: Amir Klein <[email protected]>
@amirkl94
Enable cutlass fp8 kernels
c595cbb 
Signed-off-by: Amir Klein <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mgoin mgoin Awaiting requested review from mgoin mgoin will be requested when the pull request is marked ready for review mgoin is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth will be requested when the pull request is marked ready for review tlrmchlsmth is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon will be requested when the pull request is marked ready for review WoosukKwon is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 will be requested when the pull request is marked ready for review yewentao256 is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat will be requested when the pull request is marked ready for review robertgshaw2-redhat is a code owner

@pavanimajety pavanimajety Awaiting requested review from pavanimajety pavanimajety will be requested when the pull request is marked ready for review pavanimajety is a code owner

1 more reviewer

@tomeras91 tomeras91 tomeras91 left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@amirkl94 @tomeras91