Skip to content

Commit 79a011b

Browse files
author
morelos
committed
[ET-VK] benchmarking linear_qta8a_qga4w_qta8o
# Context In order to benchmark the linear_qta8a_qga4w_qta8o operator, this diff adds functionality for int32 in the operator since Android does not support int8 operations. Functionally, this change should not be used, and this is only being provided in order to add support for int32 in android cases. # Changes Created a new benchmarking file for the linear_qta8a_qga4w_qta8o operator, which tests against various test cases. Also made a few changes to the existing operator with support for int32 along with a new variant. Differential Revision: [D77384894](https://our.internmc.facebook.com/intern/diff/D77384894/) ghstack-source-id: 292886977 Pull Request resolved: #12007
1 parent 16d9758 commit 79a011b

File tree

4 files changed

+12
-4
lines changed

4 files changed

+12
-4
lines changed

backends/vulkan/runtime/graph/ops/glsl/linear_qta8a_qga4w_qta8o_coop.glsl

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
#define NWORKERS 8
2020

2121
${define_required_extensions(DTYPE)}
22-
$if WEIGHT_STORAGE == "buffer":
22+
$if WEIGHT_STORAGE == "buffer" and WEIGHT_DTYPE == "uint8":
2323
${define_required_extensions("uint8")}
2424

2525
#extension GL_EXT_control_flow_attributes : require
@@ -28,7 +28,7 @@ layout(std430) buffer;
2828

2929
${layout_declare_tensor(B, "w", "t_out", DTYPE, OUT_STORAGE, is_scalar_array=False)}
3030
${layout_declare_tensor(B, "r", "t_mat1", DTYPE, IN_STORAGE, is_scalar_array=False)}
31-
${layout_declare_tensor(B, "r", "t_qmat2", "uint8", WEIGHT_STORAGE, is_scalar_array=False)}
31+
${layout_declare_tensor(B, "r", "t_qmat2", WEIGHT_DTYPE, WEIGHT_STORAGE, is_scalar_array=False)}
3232
${layout_declare_tensor(B, "r", "t_qparams", "float", PARAMS_STORAGE, is_scalar_array=False)}
3333
${layout_declare_tensor(B, "r", "t_input_scale", "float", "buffer", is_scalar_array=True)}
3434
${layout_declare_tensor(B, "r", "t_input_zero_point", "int", "buffer", is_scalar_array=True)}

backends/vulkan/runtime/graph/ops/glsl/linear_qta8a_qga4w_qta8o_coop.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
linear_qta8a_qga4w_qta8o_coop:
88
parameter_names_with_default_values:
99
DTYPE: int8
10+
WEIGHT_DTYPE: uint8
1011
OUT_STORAGE: texture3d
1112
IN_STORAGE: texture3d
1213
WEIGHT_STORAGE: texture2d
@@ -24,3 +25,6 @@ linear_qta8a_qga4w_qta8o_coop:
2425
- NAME: linear_qta8a_qga4w_qta8o_coop_buffer_texture2d_buffer_int8
2526
OUT_STORAGE: buffer
2627
WEIGHT_STORAGE: buffer
28+
- NAME: linear_qta8a_qga4w_qta8o_coop_texture3d_texture3d_texture2d_int32
29+
DTYPE: int32
30+
WEIGHT_DTYPE: uint32

backends/vulkan/runtime/graph/ops/glsl/linear_qta8a_qga4w_qta8o_tiled.glsl

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
#define TILE_ROWS ${TILE_ROWS}
1717

1818
${define_required_extensions(DTYPE)}
19-
$if WEIGHT_STORAGE == "buffer":
19+
$if WEIGHT_STORAGE == "buffer" and WEIGHT_DTYPE == "uint8":
2020
${define_required_extensions("uint8")}
2121

2222
#extension GL_EXT_control_flow_attributes : require
@@ -25,7 +25,7 @@ layout(std430) buffer;
2525

2626
${layout_declare_tensor(B, "w", "t_out", DTYPE, OUT_STORAGE, is_scalar_array=False)}
2727
${layout_declare_tensor(B, "r", "t_mat1", DTYPE, IN_STORAGE, is_scalar_array=False)}
28-
${layout_declare_tensor(B, "r", "t_qmat2", "uint8", WEIGHT_STORAGE, is_scalar_array=False)}
28+
${layout_declare_tensor(B, "r", "t_qmat2", WEIGHT_DTYPE, WEIGHT_STORAGE, is_scalar_array=False)}
2929
${layout_declare_tensor(B, "r", "t_qparams", "float", PARAMS_STORAGE, is_scalar_array=False)}
3030
${layout_declare_tensor(B, "r", "t_input_scale", "float", "buffer", is_scalar_array=True)}
3131
${layout_declare_tensor(B, "r", "t_input_zero_point", "int", "buffer", is_scalar_array=True)}

backends/vulkan/runtime/graph/ops/glsl/linear_qta8a_qga4w_qta8o_tiled.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
linear_qta8a_qga4w_qta8o_tiled:
88
parameter_names_with_default_values:
99
DTYPE: int8
10+
WEIGHT_DTYPE: uint8
1011
OUT_STORAGE: texture3d
1112
IN_STORAGE: texture3d
1213
WEIGHT_STORAGE: texture2d
@@ -24,3 +25,6 @@ linear_qta8a_qga4w_qta8o_tiled:
2425
- NAME: linear_qta8a_qga4w_qta8o_tiled_buffer_texture2d_buffer_int8
2526
OUT_STORAGE: buffer
2627
WEIGHT_STORAGE: buffer
28+
- NAME: linear_qta8a_qga4w_qta8o_tiled_texture3d_texture3d_texture2d_int32
29+
DTYPE: int32
30+
WEIGHT_DTYPE: uint32

0 commit comments

Comments
 (0)