Keras, PyTorch, and ONNX Accelerators with hls4ml

Platform support: ESP’s HLS flows currently target AMD/Xilinx FPGA technology mappings and are not supported by the Intel/Altera DE10-Pro SX flow. Use an RTL or Third-Party accelerator on that board.

Version-sensitive: the ESP integration sections reflect the current released tree, but the hls4ml source-generation example uses a pinned v0.2-era Vivado HLS flow and still needs end-to-end revalidation.

Open the guide

Tutorial video:


Note: The video was not recorded with the current ESP release. Use the written guide for the updated ESP integration paths and read its external hls4ml version notice before attempting source generation.


What you will learn

  • Integrate in ESP an accelerator designed in Keras/PyTorch/ONNX and generated with hls4ml
    • Generate the accelerator with hls4ml
    • Run an ESP interactive script to integrate the accelerator into ESP and to generate the Linux device driver and multiple test applications
  • Instantiate the new accelerator into an ESP SoC and test the full system with RTL simulation and on FPGA

Keras, PyTorch, and ONNX accelerator workflow


What you will need


What you can read

ESP4ML: Platform-Based Design of Systems-on-Chip for Embedded Machine Learning
Davide Giri, Kuan-lin Chiu, Giuseppe Di Guglielmo, Paolo Mantovani, Luca P. Carloni
In Proceedings of the Design, Automation and Test in Europe Conference (DATE), 2020

PAPER


What you can contribute

The ESP team welcomes external contributions and collaborations on a variety of topics including but not limited to:

  • Accelerator designs for a wide range of application domains

  • Support for more HLS tools

  • Support for accelerator design flows from domain-specific languages (DSL).

  • Power estimation flow

Check out our contributing guidelines.