make auto the default for layer config (#1016)

* make auto the default for layers

* add max_precision, not currently used

* add maximum precision in standard precision inference

* minimal handling of other types in infer_precision (e.g. for binary)

* add more checks for max precision

* fix the incorrect setting of reuse factors

* update tests to pass backend to config_from_*

* fix parameters syntax error introduced in pytest commit

* add basic type inference for embedding

* add placeholder precision inference for rnn

* fix syntax error in test_qkeras

* fix up test_trace

* don't pass auto in test_attributes

* update documentation

* update documentation (2)

* move some optimizers before infering precision type

* move up the channnels_last_converter

* put missing precision_merge logic in infer_preicion and delete, reorder optimizers

* add type inference to catapult

---------

Co-authored-by: Vladimir <[email protected]>

Author: jmitrevs

docs/api/configuration.rst

@@ -9,15 +9,18 @@ We currently support two ways of setting hls4ml's model configuration. This page
 
 .. contents:: \
 
+The Python API approach is recommended for most users as there are more utilities to help create the configuration dictionaries.
 
 **NOTE:**
 
 
 *
   One important part of ``hls4ml`` to remember is that the user is responsible for the format of the inputs.  There is no automatic formatting or normalization so this must be done in the training.
 
-*
+..
+  *
   For developers, you might also want to checkout this section: `Detailed configuration in converted hls codes <#detailed-configuration-in-converted-hls-codes>`_.
+  *Broken link*
 
 ----
 
@@ -31,11 +34,26 @@ Using hls4ml, you can quickly generate a simple configuration dictionary from a
    import hls4ml
    config = hls4ml.utils.config_from_keras_model(model, granularity='model')
 
-For more advanced and detailed configuration, you can also set them through the created dictionary. For example, to change the reuse factor:
+This python dictionary can be edited as needed. A more advanced configuration can be generated by, for example:
+
+.. code-block:: python
+
+   import hls4ml
+   config = hls4ml.utils.config_from_keras_model(
+        model,
+        granularity='name',
+        default_precision='fixed<16,6>',
+        backend='Vitis')
+
+This will include per-layer configuration based on the model. Including the backend is recommended because some configation options depend on the backend. Note, the precisions at the
+higher granularites usually default to 'auto', which means that ``hls4ml`` will try to set it automatically. Note that higher granularity settings take precendence
+over model-level settings. See :py:class:`~hls4ml.utils.config.config_from_keras_model` for more information on the various options.
+
+One can override specific values before using the configuration:
 
 .. code-block:: python
 
-   config['Model']['ReuseFactor'] = 2
+   config['LayerName']['fc1']['ReuseFactor'] = 2
 
 Or to set the precision of a specific layer's weight:
 
@@ -45,6 +63,20 @@ Or to set the precision of a specific layer's weight:
 
 To better understand how the configuration hierachy works, refer to the next section for more details.
 
+Finally, one then uses the configuration to create an hls model:
+
+.. code-block:: python
+
+  hls_model = hls4ml.converters.convert_from_keras_model(
+        model,
+        hls_config=config,
+        output_dir="my_project_dir",
+        io_type='io_stream',
+        backend='Vitis'
+    )
+
+See :py:class:`~hls4ml.converters.convert_from_keras_model` for more information on the various options.
+
 ----
 
 2. YAML Configuration file

docs/setup.rst

@@ -57,7 +57,7 @@ To run FPGA synthesis, installation of following tools is required:
 
 * Xilinx Vivado HLS 2018.2 to 2020.1 for synthesis for Xilinx FPGAs
 
-  * Vitis HLS 2022.1 or newer is required for synthesis for Xilinx FPGAs using the experimental ``Vitis`` backend.
+  * Vitis HLS 2022.2 or newer is required for synthesis for Xilinx FPGAs using the ``Vitis`` backend.
 
 * Intel Quartus 20.1 to 21.4 for the synthesis for Intel FPGAs
 

docs/status.rst

@@ -81,7 +81,7 @@ Other feature notes:
 * ``hls4ml`` is tested on Linux, and supports
    * Vivado HLS versions 2018.2 to 2020.1
    * Intel HLS versions 20.1 to 21.4
-   * Vitis HLS versions 2020.2 to 2022.2 (experimentally)
+   * Vitis HLS versions 2022.2 to 2024.1
 * Windows and macOS are not supported
 * BDT support has moved to the `Conifer <https://github.com/thesps/conifer>`__ package
 

hls4ml/backends/catapult/catapult_backend.py

@@ -110,6 +110,7 @@ def _register_flows(self):
             'catapult:inplace_stream_flatten',
             'catapult:skip_softmax',
             'catapult:fix_softmax_table_size',
+            'infer_precision_types',
         ]
         optimization_flow = register_flow('optimize', optimization_passes, requires=[init_flow], backend=self.name)
 

hls4ml/model/optimizer/__init__.py

@@ -33,9 +33,8 @@
 register_flow(
     'convert',
     [
-        'seperable_to_depthwise_and_conv',  # has to be before precision inference
-        'infer_precision_types',
         'channels_last_converter',
+        'seperable_to_depthwise_and_conv',
         'remove_transpose_before_flatten',
         'remove_nop_transpose',
         'remove_single_channel_transpose',
@@ -45,19 +44,17 @@
         'qkeras_factorize_alpha',
         'extract_ternary_threshold',
         'fuse_consecutive_batch_normalization',
+        'fuse_batch_normalization',
         'replace_multidimensional_dense_with_conv',
         'enforce_proxy_model_embedded_config',
+        'eliminate_linear_activation',
+        # many of the above optimzers need to be done before this
+        'infer_precision_types',
     ],
 )  # TODO Maybe not all QKeras optmizers belong here?
 
 register_flow(
     'optimize',
-    [
-        'eliminate_linear_activation',
-        'fuse_consecutive_batch_normalization',
-        'fuse_batch_normalization',
-        'infer_precision_types',
-        'set_precision_concat',
-    ],
+    [],
     requires=['convert'],
 )