Small Packets, Big Challenges: Enhancing Reliability in High-Speed, Low-Latency Inter-FPGA Communication

IEEE Nordic Circuits and Systems Conference (NorCAS), pp. 1–7

Abstract

Multi-FPGA systems are increasingly popular for low-power, accelerated, rapid, and high-fidelity modeling. However, their performance is often limited by FPGA-to-FPGA communication, which demands reliable, high-speed, and lowlatency links. Existing high-speed links are typically optimized for large-packet throughput, often overlooking the tight latency and bandwidth requirements of small packets. Achieving both low latency and acceptable effective bandwidth for small packets at high line rates is particularly challenging, as even moderate error rates can cause conventional approaches, such as Automatic Repeat reQuest (ARQ), to collapse under frequent retransmissions. This work addresses the latency-bandwidth trade-off for smallpacket traffic at high line rate by designing a customized serial link in an FPGA-cluster for accelerated neuroscience simulations. The design leverages a hybrid reliability architecture along with co-optimization of reliability, link, and physical layers. The reliability layer combines ARQ with the short block code$R S(31,27)_{2^{5}}$. The RS coding mitigates burst errors caused by the decision feedback equalizer required for high line-rate transmission, thereby improving the ARQ performance. The link and physical layers are designed with minimized latency and bandwidth overhead. The experimental results on two interconnected Alveo U200 boards show that the implemented link achieves 23.73 Gbps effective bandwidth with a router-to-router latency of 148 ns using 28G NRZ transceivers. To our knowledge, this is the first demonstration of a high-speed, reliable, and low-latency serial link optimized for small-packet traffic, highlighting a pathway toward more scalable and high-performance FPGA-based computing platforms.

Authors 2

1. Vida Sobhani Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair of Integrated Digital Systems and Circuit Design (IDS),Aachen,Germany

2. Tobias Gemmeke Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair of Integrated Digital Systems and Circuit Design (IDS),Aachen,Germany