Dev/perf sync

[wuhuikx]

Purpose

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[wuhuikx]

before:

[wuhuikx]

The corresponding upstream PR vllm-project#27224

[tjtanaavllm]

@wuhuikx

@ganyi1996ppo has created the PR for this feature at #745 . I have reviewed it and collected some performance data as well. It has been merged. I found that the feature also improve the performance of block-size=1 . Moreover, for overall performance block-size=1 still gives the best throughput.

[wuhuikx]

after