[ONNX] Add per channel quantization support for Onnx.QLinearConv op #3917
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vinayakdsci
reviewed
Dec 17, 2024
zjgarvey
requested changes
Dec 19, 2024
There was a problem hiding this comment.
Thanks Vivek. I think you need to modify some of the output quantization handling in the per-channel case. Maybe store a bool that tracks if we are in the per-channel case so you can reuse it for the output.
It looks like this conversion automatically fuses the input and weight quantization with the convolution, so the only thing that fuse-quantized-ops is going to do is quantize the bias (which won't work currently in the per-channel case). I think it is fine, but we won't be able to check correctness e2e until we address the per-channel quantization, unfortunately.
vivekkhandelwal1
force-pushed
the
fix-qlinearconv
branch
from
68ad21b to
32cccff
Compare
|
Hi @zjgarvey, can you please review the patch now? |
vivekkhandelwal1
force-pushed
the
fix-qlinearconv
branch
from
021fdc9 to
2c13310
Compare
zjgarvey
approved these changes
Mar 7, 2025
AmosLewis
approved these changes
Mar 9, 2025
vivekkhandelwal1
force-pushed
the
fix-qlinearconv
branch
from
2453a5b to
3d7924d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit extends the OnnxToTorch Lowering for Onnx.QLinearConv op by adding the support for per channel quantization for the weight argument.
Since the convolution operation in the downstream pipeline ("Linalg") does not support the per-channel quantization, hence we add the support by performing convolution over the dequantized input and weight and then quantizing the output.
Fixes nod-ai/SHARK-ModelDev#894.
Signed-off-by: Vivek Khandelwal [email protected]