Skip to content

[BUG] TorchDynamo frame conversion fails for autoquant because __tensor_flatten__ is not implemented for Float32Tensor #2440

New issue
New issue
Open
Open
[BUG] TorchDynamo frame conversion fails for autoquant because __tensor_flatten__ is not implemented for Float32Tensor#2440
Labels
autoquant
@vipulSharma18

Description

@vipulSharma18
vipulSharma18
opened on Jun 25, 2025

TorchDynamo tracing fails for torchao's autoquant.

Probably class Float32Tensor(TorchAOBaseTensor): needs to implement torch_flatten.

Version:

torch                     2.8.0.dev20250624+cu128
torchao                   0.11.0

Code:

    torch._dynamo.reset()
    torch._inductor.config.conv_1x1_as_mm = True
    torch._inductor.config.force_fuse_int_mm_with_mul = True
    torch._inductor.config.use_mixed_mm = True
    torch._inductor.config.coordinate_descent_tuning = True
    torch._inductor.config.coordinate_descent_check_all_directions = True
    torch._inductor.config.epilogue_fusion = True
    torch._inductor.config.prologue_fusion = True
    torch._inductor.config.benchmark_fusion = True
    torch._inductor.config.benchmark_kernel = True
    torch._inductor.config.benchmark_epilogue_fusion = True
    torch._inductor.config.benchmark_harness = True
    torch._inductor.config.combo_kernels = True
    torch._inductor.config.benchmark_combo_kernel = True
    torch._inductor.config.max_autotune_gemm = True
    torch._inductor.config.force_layout_optimization = True
    torch._inductor.config.max_autotune_pointwise = True
    torch._inductor.config.use_fast_math = True
    torch._inductor.config.cuda.use_fast_math = True
    torch._inductor.config.freezing = True
    torch._inductor.config.cpp.weight_prepack = True
    torch._inductor.config.aggressive_fusion = True

    torch.set_float32_matmul_precision("medium")
    torch.backends.cuda.enable_flash_sdp(True)
    torch.backends.cuda.matmul.allow_fp16_reduced_precision_reduction = True
    torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = True
    torch.backends.cuda.matmul.allow_fp16_accumulation = True
    torch.backends.cudnn.benchmark = True
...
    compiled_nn_model = torchao.autoquant(
        torch.compile(nn_model, mode='max-autotune', dynamic=False, fullgraph=True),
        example_input=dummy,
        qtensor_class_list=ALL_AUTOQUANT_CLASS_LIST,
        set_inductor_config=False,
    )

Error in TorchDynamo logs:

W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] WON'T CONVERT _quantized_linear_op /app/.venv/lib/python3.12/site-packages/torchao/quantization/autoquant.py line 817
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] ========== TorchDynamo Stack Trace ==========
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] Traceback (most recent call last):
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1272, in __call__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     result = self._inner_convert(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]              ^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 629, in __call__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return _compile(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1164, in _compile
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     raise InternalTorchDynamoError(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1111, in _compile
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     guarded_code = compile_inner(code, one_graph, hooks, transform)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_utils_internal.py", line 97, in wrapper_function
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return function(*args, **kwargs)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 793, in compile_inner
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return _compile_inner(code, one_graph, hooks, transform)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 832, in _compile_inner
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     out_code = transform_code_object(code, transform)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/bytecode_transformation.py", line 1424, in transform_code_object
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     transformations(instructions, code_options)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 267, in _fn
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return fn(*args, **kwargs)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 753, in transform
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     tracer.run()
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 3497, in run
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     super().run()
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     while self.step():
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]           ^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self.dispatch_table[inst.opcode](self, inst)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2294, in LOAD_ATTR
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self._load_attr(inst)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2282, in _load_attr
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     result = BuiltinVariable(getattr).call_function(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builtin.py", line 1189, in call_function
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return handler(tx, args, kwargs)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builtin.py", line 822, in <lambda>
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     tx, [v.realize() for v in args], kwargs
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]          ^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/lazy.py", line 67, in realize
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self._cache.realize()
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/lazy.py", line 32, in realize
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self.vt = builder.VariableBuilder(tx, self.source)(self.value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 440, in __call__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     vt = self._wrap(value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]          ^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 681, in _wrap
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return self.wrap_tensor(value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 1979, in wrap_tensor
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self.assert_not_wrapped_by_this_graph(value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 1860, in assert_not_wrapped_by_this_graph
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     if is_fake(value) and maybe_get_fake_mode(value) is self.tx.fake_mode:
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]        ^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 193, in is_fake
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     attrs, _ = type(x).__tensor_flatten__(x)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torchao/utils.py", line 570, in __tensor_flatten__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     raise NotImplementedError("Subclasses must implement __tensor_flatten__")
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] torch._dynamo.exc.InternalTorchDynamoError: NotImplementedError: Subclasses must implement __tensor_flatten__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] 
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] from user code:
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]    File "/app/.venv/lib/python3.12/site-packages/torchao/quantization/autoquant.py", line 823, in _quantized_linear_op
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     w_qtensor.weight,
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] 
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] Traceback (most recent call last):
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1272, in __call__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     result = self._inner_convert(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]              ^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 629, in __call__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return _compile(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1164, in _compile
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     raise InternalTorchDynamoError(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1111, in _compile
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     guarded_code = compile_inner(code, one_graph, hooks, transform)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_utils_internal.py", line 97, in wrapper_function
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return function(*args, **kwargs)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 793, in compile_inner
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return _compile_inner(code, one_graph, hooks, transform)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 832, in _compile_inner
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     out_code = transform_code_object(code, transform)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/bytecode_transformation.py", line 1424, in transform_code_object
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     transformations(instructions, code_options)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 267, in _fn
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return fn(*args, **kwargs)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 753, in transform
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     tracer.run()
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 3497, in run
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     super().run()
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1363, in run
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     while self.step():
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]           ^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 1267, in step
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self.dispatch_table[inst.opcode](self, inst)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2294, in LOAD_ATTR
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self._load_attr(inst)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2282, in _load_attr
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     result = BuiltinVariable(getattr).call_function(
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builtin.py", line 1189, in call_function
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return handler(tx, args, kwargs)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builtin.py", line 822, in <lambda>
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     tx, [v.realize() for v in args], kwargs
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]          ^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/lazy.py", line 67, in realize
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self._cache.realize()
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/lazy.py", line 32, in realize
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self.vt = builder.VariableBuilder(tx, self.source)(self.value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 440, in __call__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     vt = self._wrap(value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]          ^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 681, in _wrap
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     return self.wrap_tensor(value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]            ^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 1979, in wrap_tensor
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     self.assert_not_wrapped_by_this_graph(value)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_dynamo/variables/builder.py", line 1860, in assert_not_wrapped_by_this_graph
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     if is_fake(value) and maybe_get_fake_mode(value) is self.tx.fake_mode:
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]        ^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 193, in is_fake
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     attrs, _ = type(x).__tensor_flatten__(x)
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]   File "/app/.venv/lib/python3.12/site-packages/torchao/utils.py", line 570, in __tensor_flatten__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     raise NotImplementedError("Subclasses must implement __tensor_flatten__")
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] torch._dynamo.exc.InternalTorchDynamoError: NotImplementedError: Subclasses must implement __tensor_flatten__
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] 
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] from user code:
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]    File "/app/.venv/lib/python3.12/site-packages/torchao/quantization/autoquant.py", line 823, in _quantized_linear_op
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339]     w_qtensor.weight,
W0625 03:25:48.966000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py:1339] 
I0625 03:25:49.138000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/__init__.py:118] torch._dynamo.reset
I0625 03:25:49.138000 127224 .venv/lib/python3.12/site-packages/torch/_dynamo/__init__.py:151] torch._dynamo.reset_code_caches
I0625 03:25:49.140000 127224 .venv/lib/python3.12/site-packages/torch/_utils_internal.py:122] dynamo _convert_frame_assert._compile: {'co_name': '_quantized_linear_op', 'frame_id': 0, 'compile_id': '0/0', 'co_filename': '/app/.venv/lib/python3.12/site-packages/torchao/quantization/autoquant.py', 'co_firstlineno': 580, 'cache_size': 0, 'accumulated_cache_size': 0}

Metadata

Metadata

Assignees

No one assigned

    Labels

    autoquant

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions