Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark Hugging Face models via Optimum integration #8294

Open
guangy10 opened this issue Feb 6, 2025 · 1 comment
Open

Benchmark Hugging Face models via Optimum integration #8294

guangy10 opened this issue Feb 6, 2025 · 1 comment
Assignees
@guangy10
Labels
module: benchmark Issues related to the benchmark infrastructure triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Milestone
0.6.0

Comments

@guangy10
Copy link
Contributor

guangy10 commented Feb 6, 2025 •
edited by pytorch-bot bot
Loading

🚀 The feature, motivation and pitch

Model sourcing: huggingface/transformers
Recipes (pte generation and distribution): optimum-executorch

This work is critical as we start adding and validating recipes for non-cpu delegates.

Note this task is to track the infra work. Individual model and delegate enablement is out of scope and will be tracked separately.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @huydhn @kirklandsign @shoumikhin
@guangy10 guangy10 added feature module: benchmark Issues related to the benchmark infrastructure labels Feb 6, 2025
@guangy10 guangy10 self-assigned this Feb 6, 2025
@digantdesai digantdesai added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Feb 7, 2025
@guangy10 guangy10 moved this to Ready in ExecuTorch Benchmark Feb 10, 2025
@guangy10 guangy10 added this to the 0.6.0 milestone Feb 10, 2025
This was referenced Feb 11, 2025
Open
Open
@jackzhxng jackzhxng removed the feature label Feb 21, 2025
@guangy10
Copy link
Contributor Author

Start with CI integration: #8630

This was referenced Feb 24, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@guangy10 guangy10

Labels
module: benchmark Issues related to the benchmark infrastructure triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
ExecuTorch Benchmark
Status: Ready
Milestone
0.6.0
Development

No branches or pull requests
3 participants
@digantdesai @jackzhxng @guangy10