Is weight sharing in Vulkan backend supported?

[Dadada0502]

🐛 Describe the bug

In the v0.6.0 release notes, I noticed that for XNNPack the build option ENABLE_XNNPACK_WEIGHTS_CACHE was introduced to enable weight sharing across multiple methods within the same model. However, when following the Vulkan tutorial and documentation, there doesn’t appear to be an equivalent option for weight sharing.

Does the Vulkan backend in v0.6.0 support weight sharing (either through a CMake build flag or a runtime option)?

If not, are there plans to introduce weight sharing or similar weight-caching features in future Vulkan releases?

In the meantime, what optimizations or best practices (e.g., memory layout tweaks, shader tuning) would you recommend to improve Vulkan performance?

Versions

executorch v0.6.0

cc @SS-JIA @manuelcandales @cbilgin

[SS-JIA]

Hello @Dadada0502, thanks for your interest in the Vulkan delegate!

Does the Vulkan backend in v0.6.0 support weight sharing (either through a CMake build flag or a runtime option)?

If not, are there plans to introduce weight sharing or similar weight-caching features in future Vulkan releases?

Unfortunately, weight sharing is not enabled in Vulkan at the moment. Although I am not currently working on the feature, it is on my radar and I can try to work it into the list of features to enable by ExecuTorch's 1.0 release. Is there a specific model that you wanted to use weight sharing with? It would be helpful to know more about your specific use case.

In the meantime, what optimizations or best practices (e.g., memory layout tweaks, shader tuning) would you recommend to improve Vulkan performance?

In terms of memory layouts, the export logic to try and manage that automatically for you. The logic for this is split between op_registry.py which encodes operators' preferences for specific storage types and memory layouts, and tag_memory_meta_pass.py which makes sure inputs to operators are in the operator's preferred layout.

Regarding shader tuning, there are two things you can try to experiment with the performance of ET-VK

1. Try modifying the local work group size of key compute shaders; different local work group size configurations may work better for different devices. You can modify this by changing how local work group size is selected in callsites such as this
2. If you are familiar with writing and optimizing compute shaders, I would also invite you to try writing a faster compute shader for key operators that work better on your device/platform than what we currently have in our code-base. I would be happy to expand our compute shader library, and our C++ logic can dispatch to your new compute shader when certain conditions are met.

As an aside, I would also recommend joining the ExecuTorch project's discord channel. We have a vulkan channel there to discuss the Vulkan delegate, and the chat format facilitates faster back and forth on most topics. It would be great to continue a conversation on optimization there!