Communication-efficient Decentralised Federated Learning via Low Huffman-coded Delta Quantization Scheme

International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 31–36

Abstract

Federated Learning (FL) revolutionizes distributed machine learning, enabling clients to learn collaboratively while keeping data private. In contrast, Decentralized Federated Learning (DFL) offers direct communication between clients without a central server, improving fault tolerance and network efficiency, but communication overhead remains a challenge. To address this, we propose a new scheme called Low Huffman-coded Delta Quantization (LHDQ) which achieves a remarkable quantization rate of $\frac{5}{3}$ bits per parameter. We evaluate LHDQ within the DFL architecture under two proposed transmission protocols and compare it against conventional quantization schemes under various communication channel conditions. Despite a slight reduction in accuracy, LHDQ offers compelling advantages as alleviating communication bottlenecks, reducing transmitted bits, and accelerating training and convergence processes.

Authors 4

1. Mahdi Barhoush Aachen Chair of Information Theory and Data Analytics

   RWTH Aachen University

   Affiliation as printed

   RWTH University,Chair of Information Theory and Data Analytics (INDA),Aachen,Germany

2. Ahmad Ayad Aachen Chair of Information Theory and Data Analytics

   RWTH Aachen University

   Affiliation as printed

   RWTH University,Chair of Information Theory and Data Analytics (INDA),Aachen,Germany

3. Mohammad Kohankhaki Aachen Chair of Information Theory and Data Analytics

   RWTH Aachen University

   Affiliation as printed

   RWTH University,Chair of Information Theory and Data Analytics (INDA),Aachen,Germany

4. Anke Schmeink Aachen Chair of Information Theory and Data Analytics

   RWTH Aachen University

   Affiliation as printed

   RWTH University,Chair of Information Theory and Data Analytics (INDA),Aachen,Germany