Add IntxUnpackedTensor #2732
Add IntxUnpackedTensor #2732
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2732
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 93948a4 with merge base 6cfa477 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
| block_size: the block size for quantization, representing the granularity, for example groupwise quantization will have block_size (1, group_size) | ||
| """ | ||
|
|
||
| tensor_data_attrs = ["int_data", "scale", "zero_point"] |
There was a problem hiding this comment.
btw if you update these to tensor_data_names and tensor_attribute_names you'll be able to remove some of the implementations, see docs in https://github.com/pytorch/ao/pull/2710/files#diff-d2a11602a79e83305208472f1abe6a4106f02ce62a7f9524007181813863fcf6R687, example: #2738
There was a problem hiding this comment.
I can still override the behavior in TorchAOBaseTensor, right?
For example, it looks like aten._to_copy.default gets auto-populated, but I want to define its dtype variant in addition to device variant.
There was a problem hiding this comment.
this should be working, I haven't actively tested this behavior though, I'll try to add a test for this
| ) | ||
|
|
||
| @classmethod | ||
| def from_float( |
There was a problem hiding this comment.
nit: we are standardizing on from_hp now
There was a problem hiding this comment.
What does hp stand for?
| @@ -30,3 +30,8 @@ class PackingFormat(str, Enum): | |||
| preshuffled is referring to the preshuffled format used by fbgemm kernels | |||
| """ | |||
| PRESHUFFLED = "preshuffled" | |||
|
|
|||
| """ | |||
| Unpacked means the subbyte quantized data is stored as int8 | |||
There was a problem hiding this comment.
is this int only? we could be more specific and say UnpackedToInt8
There was a problem hiding this comment.
Sure, I can make the format UNPACKED_TO_INT8
torchao/quantization/quant_api.py
Outdated
| @@ -2060,6 +2061,8 @@ class IntxWeightOnlyConfig(AOBaseConfig): | |||
| mapping_type: MappingType = MappingType.SYMMETRIC | |||
| scale_dtype: Optional[torch.dtype] = None | |||
| layout: Layout = QDQLayout() | |||
| packing_format: PackingFormat = PackingFormat.UNPACKED | |||
| VERSION: int = 1 | |||
There was a problem hiding this comment.
nit: we updated the name to version
metascroy
added
the
topic: not user facing
No description provided.