-
-
Notifications
You must be signed in to change notification settings - Fork 10k
[Model] Officially support Emu3 with Transformers backend #21319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request officially adds support for the Emu3 model by redirecting it to the Transformers backend, and updates the documentation accordingly. The code changes are logical and well-structured. My main feedback is to add a corresponding test case for the newly supported model to ensure its correctness and prevent future regressions, as is standard practice in this repository.
Signed-off-by: Harry Mellor <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, hope that this would encourage more people to consider Transformers backend before opening their own PRs!
PTAL at multi-modal tests failure |
It's a weight mapping issue. We've had a TODO in the code for a while to simplify the weight mapping so I've made #21382 to do that ahead of this PR. |
Signed-off-by: Harry Mellor <[email protected]>
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Harry Mellor <[email protected]>
Fixing the |
Signed-off-by: Harry Mellor <[email protected]>
The failing mapper test was using I have modified the test to ignore buffers that it may find in checkpoints. |
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: avigny <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: shuw <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: x22x22 <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: Paul Pak <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: Boyuan Feng <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]> Signed-off-by: Diego-Castan <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]>
…ct#21319) Signed-off-by: Harry Mellor <[email protected]>
This PR adds an explicit redirect from
Emu3ForConditionalGeneration
toTransformersForMultimodalLM
. This means that:In the docs I have added a paragraph explaining what it means for a model to be officially supported via the Transformers backend.
Closes #11008