-
-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Spec Decode] Add Script for converting HF Eagle checkpoint to vLLM compatible checkpoint #11866
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: root <Sourashis Roy>
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
@LiuXiaoxuanPKU / @llsj14 PTAL when you get a chance |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 👍
'--baseline_ckpt_file_lm_head', | ||
type=str, | ||
help="Path to the baseline model checkpoint file containing the " | ||
"weights for lm_head") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be also nice if you inform the baseline ckpt file should be 'safetensor'. Because in this script, the load_file
function is for only safetensors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the comment.
Signed-off-by: root <Sourashis Roy>
#11126