Skip to content

Conversation

@mohitmundhragithub
Copy link
Contributor

No description provided.

@mohitmundhragithub mohitmundhragithub requested review from a team and anhappdev as code owners August 26, 2025 05:41
@github-actions
Copy link

github-actions bot commented Aug 26, 2025

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

…lemented performance benchmark for LLM pipeline
…y input and issue_query only handles output tokens
@farook-edev farook-edev changed the title Feat llm LLM pipeline implementation Sep 2, 2025
@farook-edev farook-edev linked an issue Sep 2, 2025 that may be closed by this pull request
Copy link
Contributor Author

@mohitmundhragithub mohitmundhragithub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@sonarqubecloud
Copy link

@farook-edev farook-edev marked this pull request as ready for review October 31, 2025 08:56
@farook-edev
Copy link
Contributor

Regarding IOS CI issue, the problem was in 2 parts:

  1. Eigen enabled exceptions regardless of fno-exceptions
  2. Tensorflow -> XNNPack -> FP16 -> math.h was incompatible with x86_64 simulator

1 was resolved by adding a patch that -for the time being- creates a macro that forces exceptions to be off. This macro is enabled only for IOS builds
NOTE: This will be unnecessary once Tensorflow is updated, since newer versions seem to drop Eigen as a dependency

2 was resolved by getting the same version of Tensorflow 2.18.0->XNNPack->FP16 and applying a patch that removes math.h as a dependency.
NOTE: THIS WILL REDUCE IOS PERFORMANCE according to some comments on github, and will be unnecessary once XNNPack is upgraded, since XNNPack reportedly dropped FP16 dependency early this year.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Master issue: LLM Benchmark

6 participants