Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

report regression in executorch-0.5.0 #14

Closed
guangy10 opened this issue Feb 3, 2025 · 1 comment
Closed

report regression in executorch-0.5.0 #14

guangy10 opened this issue Feb 3, 2025 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@guangy10
Copy link
Collaborator

guangy10 commented Feb 3, 2025

ExecuTorch 0.5.0 was released: https://github.com/pytorch/executorch/releases/tag/v0.5.0. However, this repo is new and there is no cross-repo integration tests, so there may be some regressions in 0.5.0. However, since 0.5.0 is out, there is no way to fix it back in 0.5.0, we can only target to get all known integration issues fixed by the upcoming 0.6.0 in Mar, 2025. The good thing is that we start pinning optimum-executorch to ExecuTorch CI in Run Hugging Face models via Optimum on CI pytorch/executorch#8630

If you run into issue using executorch-0.5.0, you can be unblocked by trying either:

  • downgrade to 0.4.0 by running pip install executorch==0.4.0
  • install executorch from main branch.
@guangy10
Copy link
Collaborator Author

Notice an issue when running Qwen2.5 with executorch 0.5.0. Works fine with 0.4.0

[tensor_util.h:713] Tensors do not match: numel=(2048,  1), dim=(1, 2)
[tensor_util.h:720]     size(0): (2048, 1)
[op_le.cpp:29] Check failed (tensors_have_same_shape(a, b)):
[method.cpp:1072] KernelCall failed at instruction 0:18 in operator aten::le.Tensor_out: 0x12
[method.cpp:1078] arg 0 with type id 1
[method.cpp:1078] arg 1 with type id 1
[method.cpp:1078] arg 2 with type id 1
[method.cpp:1078] arg 3 with type id 1
Traceback (most recent call last):
  File "/Users/guangyang/optimum-executorch/optimum/executorch/modeling.py", line 360, in text_generation
    generated_tokens = self.generate(
                       ^^^^^^^^^^^^^^
  File "/Users/guangyang/optimum-executorch/optimum/executorch/modeling.py", line 301, in generate
    logits = self.forward(
             ^^^^^^^^^^^^^
  File "/Users/guangyang/optimum-executorch/optimum/executorch/modeling.py", line 114, in forward
    return self.model.forward((input_ids, cache_position))[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: method->execute() failed with error 0x12

@guangy10 guangy10 changed the title Add support for executorch-0.5.0 bugbash for executorch-0.5.0 integration Feb 27, 2025
@guangy10 guangy10 added the bug Something isn't working label Feb 27, 2025
@guangy10 guangy10 changed the title bugbash for executorch-0.5.0 integration bugbash regression in executorch-0.5.0 Feb 28, 2025
@guangy10 guangy10 changed the title bugbash regression in executorch-0.5.0 report regression in executorch-0.5.0 Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant