Add LayerNorm support for Vivado #1110
Conversation
|
Any update on the merge ? |
|
First in line on my PR review TODO list, I expect early next week to have time for this. |
Thank you very much ! |
|
pre-commit.ci autofix |
JanFSchulte
added
please test
|
@vloncar I hope this goes in the direction of what you had in mind for the performance validation. I ran synthesis in Vitis 2023.1, 2024.1, and 2025.1 for different input sizes to the LayerNorm and plotted FFs, LUTs, DSPs, BRAM, latency, and II as a function of that input size. 2024.1 and 2025.1 are basically identical, whereas 2023.1 uses a bit less resources but has worse latency. This is the the default ap_fixed<16.6> and the default target part. 
|
|
Thanks, looks good. Do all reports say the timing is met? (No scheduling warnings etc, the clock uncertainty is met etc) |
|
I did not observe any warning about the scheduling. The timing results look all very similar: |
| static const unsigned dim = CONFIG_T::n_in / CONFIG_T::seq_len; | ||
| data_T in_val[dim]; | ||
| res_T outval[dim]; | ||
| // Use a function_instantiate in case it helps to explicitly optimize unchanging weights/biases |
There was a problem hiding this comment.
It does not. I think this can be removed from new code.
test/pytest/test_layernorm.py
Outdated
| hls_model = hls4ml.converters.convert_from_keras_model( | ||
| custom_epsilon_model, backend=backend, hls_config=custom_config, io_type='io_parallel', output_dir=output_dir | ||
| ) | ||
| hls_model.compile() |
There was a problem hiding this comment.
This test would be faster if we used hls_model.write(), or completely skip the step of writing/linking. We don't use it here. The later accuracy test checks if the produced code is compilable
test/pytest/test_layernorm.py
Outdated
| # Predict | ||
| y_keras = model.predict(data).flatten() | ||
| y_hls = hls_model.predict(data).flatten() | ||
| np.testing.assert_allclose(y_keras, y_hls, rtol=0, atol=atol, verbose=True) |
There was a problem hiding this comment.
why is atol a global variable?
hls4ml/converters/pytorch/core.py
Outdated
|
|
||
| if not ((len(input_shapes[0])) == 3): | ||
| raise Exception( | ||
| 'input size is not currently supported by hls4ml; ' |
There was a problem hiding this comment.
Would be good to say Input shape <some shape> is not supported, only ...
| @@ -44,6 +44,25 @@ def transform(self, model, node): | |||
| node.get_output_variable().shape = input_shape | |||
| dim_names = [f'N_INPUT_{i}_{node.index}' for i in range(1, len(input_shape) + 1)] | |||
| node.get_output_variable().dim_names = dim_names | |||
| elif ( | |||
| isinstance(node, LayerNormalization) | |||
| and not model.config.config['HLSConfig']['Model']['ChannelsLastConversion'] == "off" | |||
There was a problem hiding this comment.
With the proposed change in #1352 we'll never get to off check here.
JanFSchulte
added
please test
JanFSchulte
added
please test
JanFSchulte
added
please test
Description
This PR adds support for Layer Normalization using either Keras or PyTorch with the Vivado backend in
io_parallelmode.This implementation uses a lookup table for inverse square root; the inputs to the lookup table follow a logarithmic distribution for better accuracy.
Tests have been added for both Keras and Pytorch parsing.
Credit is due to @Ethan0Jiang and @LostEcho365 (Zhixing Jiang and Dennis Yin) for their Vivado implementation and Keras parsing support; my contributions were making a change to the inverse square root lookup table implementation, implementing PyTorch support, and adding unit tests. (Here's a link to their pre-print.) The original code authors have given permission for their code to be merged into hls4ml.
Linked issue: #1109
Type of change
Tests
Two unit tests added:
test/pytest/test_layernorm.pyandtest/pytest/test_layernorm_pytorch.pyChecklist
pre-commiton the files I edited or added.