hls4ml-gravnet

This repository contains the High-Level Synthesis (HLS) headers and Python auxiliaries required to synthesize the quantized GravNet model from quantized-gravnet using hls4ml for real-time inference on FPGAs.

Key Features & Requirements

• Utilizes the hls4ml Extension API to convert models from Keras to HLS.
• Experimental Fork: This project was initially tested with a specific experimental fork of hls4ml located here. This fork includes necessary changes to support:
  • Synthesizing hls4ml with the Coyote accelerator backend.
  • Compatibility with Vitis/Vivado 2025.2, with Coyote.
  • Processing two vertices simultaneously per clock cycle.
  • Supporting multiple outputs with Coyote.

Any feedback for running synthesis with the hls4ml upstream is highly welcome.

Getting Started

Prerequisites

uv is used as the package manager. Make sure it is available on your system before proceeding.

Installation

To create a virtual environment and install the hls4ml_gravnet package, run:

uv sync

If you want to install it as an editable package for active development:

uv pip install -e .

If you already have a project and want to add this package to your existing virtual environment:

uv add git+[https://github.com/morunner/hls4ml-gravnet.git](https://github.com/morunner/hls4ml-gravnet.git)

Usage

Here is an example of converting the QGravNetFactory model to HLS using the Vitis (or CoyoteAccelerator) backend.

import hls4ml
from hls4ml_gravnet.hls4ml_extension.register_extensions import register_extensions

# Register the custom GravNet layers
register_extensions(backend='Vitis') # Or 'CoyoteAccelerator'

# Configure HLS from the Keras model
hls_config = hls4ml.utils.config_from_keras_model(model)

# Note: Tune precisions and reuse factors in the hls_config here as needed.

# Convert the model to an HLS project
hls_model = hls4ml.converters.convert_from_keras_model(
    model=model,
    hls_config=hls_config,
    backend='Vitis',
    output_dir='my_hls_project'
)

# Compile the HLS model for C-simulation
hls_model.compile()

# Synthesize the RTL hardware
hls_model.build()

References

This project implements layers from the original GravNet architecture described in:

Learning representations of irregular particle-detector geometry with distance-weighted graph networks Shah Rukh Qasim, Jan Kieseler, Yutaro Iiyama, Maurizio Pierini Eur. Phys. J. C79 (2019) no.7, 608. [DOI]

@article{Qasim:2019otl,
      author         = "Qasim, Shah Rukh and Kieseler, Jan and Iiyama, Yutaro and
                        Pierini, Maurizio",
      title          = "{Learning representations of irregular particle-detector
                        geometry with distance-weighted graph networks}",
      journal        = "Eur. Phys. J.",
      volume         = "C79",
      year           = "2019",
      number         = "7",
      pages          = "608",
      doi            = "10.1140/epjc/s10052-019-7113-9",
      eprint         = "1902.07987",
      archivePrefix  = "arXiv",
      primaryClass   = "physics.data-an",
      SLACcitation   = "%%CITATION = ARXIV:1902.07987;%%"
}

The hardware implementations in this repository succeeded thanks to the foundational work and architectural designs from:

Real-Time Graph-based Point Cloud Networks on FPGAs via Stall-Free Deep Pipelining Marc Neu, Isabel Haide, Timo Justinger, Till Rädler, Valdrin Dajaku, Torben Ferber, Jürgen Becker arXiv preprint (2025). [arXiv:2507.05099]

@misc{neu2025,
      title={Real-Time Graph-based Point Cloud Networks on FPGAs via Stall-Free Deep Pipelining},
      author={Marc Neu and Isabel Haide and Timo Justinger and Till Rädler and Valdrin Dajaku and Torben Ferber and Jürgen Becker},
      year={2025},
      eprint={2507.05099},
      archivePrefix={arXiv},
      primaryClass={eess.SP},
      url={[https://arxiv.org/abs/2507.05099](https://arxiv.org/abs/2507.05099)},
}