Skip to content

Add torchao kernels to xcframework #10694

Open
@metascroy

Description

@metascroy

🚀 The feature, motivation and pitch

We can build low-bit kernels on Macbook (https://github.com/pytorch/executorch/tree/main/examples/models/llama#running-with-low-bit-kernels), but we don't currently build these as part of the main ExecuTorch CMakeLists.txt.

Instead they are built inside the runner code here: https://github.com/pytorch/executorch/blob/main/examples/models/llama/CMakeLists.txt#L118-L137

In this task, we want to add an option to build this kernel library to the ET CMakeLists.txt (https://github.com/pytorch/executorch/blob/main/CMakeLists.txt) guarded under EXECUTORCH_BUILD_KERNELS_TORCHAO, and add them to the xcframeworks (https://github.com/pytorch/executorch/blob/main/scripts/build_apple_frameworks.sh).

This will let us run low-bit kernels on the iOS app.

cc @larryliu0820 @jathu @kimishpatel @jerryzh168 @shoumikhin @cbilgin

Metadata

Metadata

Assignees

Labels

good first issueGood for newcomersmodule: build/installIssues related to the cmake and buck2 builds, and to installing ExecuTorchmodule: iosIssues related to iOS code, build, and executionmodule: quantizationIssues related to quantizationtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions