[WIP][high-risk][Step 1]Refactor compressor and quantizer#2039
[WIP][high-risk][Step 1]Refactor compressor and quantizer#2039wenhuach21 wants to merge 114 commits into
Conversation
for more information, see https://pre-commit.ci
…to refine_calib # Conflicts: # auto_round/algorithms/pipeline.py # auto_round/algorithms/quantization/base.py
for more information, see https://pre-commit.ci
|
issue repeated log /home/wenhuach/miniforge3/envs/autoround/bin/python /home/wenhuach/auto-round/auto_round/main.py /models/Qwen3-0.6B --tasks lambada_openai --algs awq,auto-round --seqlen 512 |
|
2 no loss print 2026-07-10 17:02:13 INFO mappings.py L560: AWQ resolved 84 smooth-balance mappings (3 per block × 28 blocks). 3 need to check gradient accumulate step passing correctly 4 refine zero_shot |
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
|
|
||
| @torch.no_grad() | ||
| def quantize_block(self, ctx) -> dict: | ||
| def quantize_block(self, block, fp_inputs, input_others, fp_outputs, q_inputs, block_ctx) -> dict: |
There was a problem hiding this comment.
q_inputs default none
…to refine_calib # Conflicts: # auto_round/algorithms/base.py # auto_round/algorithms/quantization/base.py
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
wenhuach21
changed the title
for more information, see https://pre-commit.ci
| block, sync_gradients = setup_ddp_if_needed_(self, block, device_manager.device_list) | ||
| index_sampler = IndexSampler(nsamples, global_batch_size) | ||
| batch_size = self.batch_size | ||
| block_fwd = self.compressor.block_forward |
There was a problem hiding this comment.
decouple, using bind instead
…to refine_calib # Conflicts: # auto_round/calibration/base.py
for more information, see https://pre-commit.ci
|
/azp run Unit-Test-CUDA-AutoRound |
|
Azure Pipelines successfully started running 1 pipeline(s). |
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
|
/azp run Unit-Test-CUDA-AutoRound |
|
Azure Pipelines successfully started running 1 pipeline(s). |
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
Description
Please briefly describe your main changes, the motivation.
Type of Change
Bug fix
Related Issues
Fixes or relates to #
Checklist Before Submitting
/azp run Unit-Test-CUDA-AutoRound.must fix
1 mllm change to batch_size 1, change gradient correspoindly
2 block forward torch compile
gradient accumulate step warning in cli
verify algs name
/models/Qwen3-8B --tasks lambada_openai --algs haha --seqlen 512 --iters 1 --gradient_accumulate_steps 2
decouple calib with compressor
current calib set some values via calib state in compressor, which is ugly
issue repeated log
/home/wenhuach/miniforge3/envs/autoround/bin/python /home/wenhuach/auto-round/auto_round/main.py /models/Qwen3-0.6B --tasks lambada_openai --algs awq,auto-round --seqlen 512
2026-07-10 17:01:26 INFO main.py L292: start to quantize /models/Qwen3-0.6B
2026-07-10 17:01:26 INFO config.py L53: enable_opt_rtn is turned on, set --disable_opt_rtn for higher speed at the cost of accuracy.
2026-07-10 17:01:26 INFO config.py L53: enable_opt_rtn is turned on, set --disable_opt_rtn for higher speed at the cost of accuracy.
attetnion_mask(pad_token) has not passed to alg
issue no loss print
2026-07-10 17:02:13 INFO mappings.py L560: AWQ resolved 84 smooth-balance mappings (3 per block × 28 blocks).
2026-07-10 17:02:13 INFO mappings.py L423: Using registered AWQ mappings for Qwen3ForCausalLM.
2026-07-10 17:02:13 INFO mappings.py L560: AWQ resolved 84 smooth-balance mappings (3 per block × 28 blocks).
2026-07-10 17:02:13 INFO base.py L201: AWQ: resolved 84 mappings across 28 blocks.
Quantizing model.layers.0: 0%| | 0/28 [00:00<?, ?it/s]2026-07-10 17:02:52 INFO device.py L1450: 'peak_ram': 2.54GB, 'peak_vram': 1.25GB
Quantizing model.layers.1: 4%|▌ | 1/28 [00:39<17:35, 39.10s/it]2026-07-10 17:03:24 INFO device.py L1450: 'peak_ram': 2.57GB, 'peak_vram': 1.37GB
Quantizing model.layers.2: 7%|█▏ | 2/28 [01:10<15:04, 34.80s/it]
move attention_mask to quantize_block
it may be better just use inherit instead of mixin for mllm and dissfusion
adapt mxfp4/nvfp4/diffusion opt-rtn to rtn