perf: add dsv2 performance change#1370
Open
baominghelly wants to merge 1 commit into
Open
Conversation
baominghelly
force-pushed
the
feat/dsv2-perf_optimization
branch
from
98fdf2e to
2d5e163
Compare
wooway777
changed the title
wooway777
changed the title
wooway777
reviewed
Jul 7, 2026
| } | ||
| assert(dim_shape == out->shape()[dim]); | ||
|
|
||
| if (out->device().getType() == Device::Type::HYGON) { |
Collaborator
There was a problem hiding this comment.
这个判定是非得加么？如果这么做更好，能否扩展到其他平台？
wooway777
reviewed
Jul 7, 2026
| if (expert >= 0 && static_cast<size_t>(expert) < num_experts) { | ||
| const T *down = reinterpret_cast<const T *>(down_weights[expert]) + h * intermediate_size; | ||
| acc += to_float<T>(intermediate[route * intermediate_size + j]) * to_float<T>(down[j]); | ||
| #if defined(ENABLE_HYGON_API) |
Collaborator
There was a problem hiding this comment.
相关优化是只有海光有效果，还是其他平台也能复用？
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cat.cc海光上将大量小Memcpy改成按输出切片执行 copy_from，减少MLA拼接开销，仅海光平台deepseek_moe_nvidia.cu针对不同token规模增加多条MoE计算路径，优化Decode和Prefill，NVIDIA和海光共享文件，部分优化仅海光paged_attention_mla_hd576_v512.cu海光Paged MLA Decode默认split数从4调整为8, 仅海光平台Test evidence
海光平台
deepseek moe
cat
paged attention
英伟达平台
deepseek moe
cat
paged attention
