-
Notifications
You must be signed in to change notification settings - Fork 29
LLM pipeline implementation #1040
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…re pipeline cannot handle an input size larger than the max prefill size
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
…lemented performance benchmark for LLM pipeline
…y input and issue_query only handles output tokens
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.
|
|
Regarding IOS CI issue, the problem was in 2 parts:
1 was resolved by adding a patch that -for the time being- creates a macro that forces exceptions to be off. This macro is enabled only for IOS builds 2 was resolved by getting the same version of Tensorflow 2.18.0->XNNPack->FP16 and applying a patch that removes math.h as a dependency. |



No description provided.