FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning

arXiv (Cornell University)

Abstract

Federated Learning (FL) heavily depends on label quality for its performance. However, the label distribution among individual clients is always both noisy and heterogeneous. The high loss incurred by client-specific samples in heterogeneous label noise poses challenges for distinguishing between client-specific and noisy label samples, impacting the effectiveness of existing label noise learning approaches. To tackle this issue, we propose FedFixer, where the personalized model is introduced to cooperate with the global model to effectively select clean client-specific samples. In the dual models, updating the personalized model solely at a local level can lead to overfitting on noisy data due to limited samples, consequently affecting both the local and global models' performance. To mitigate overfitting, we address this concern from two perspectives. Firstly, we employ a confidence regularizer to alleviate the impact of unconfident predictions caused by label noise. Secondly, a distance regularizer is implemented to constrain the disparity between the personalized and global models. We validate the effectiveness of FedFixer through extensive experiments on benchmark datasets. The results demonstrate that FedFixer can perform well in filtering noisy label samples on different clients, especially in highly heterogeneous label noise scenarios.

Authors 7

1. Xinyuan Ji Aachen

   Leiden University · Xi'an Jiaotong University

   Affiliation as printed

   Leiden University

   Xi'an Jiaotong University

2. Zhaowei Zhu

   Doctors Hospital

   Affiliation as printed

   Docta

3. Xi Wei

   Xi'an Jiaotong University

   Affiliation as printed

   Xi'an Jiaotong University

4. Olga Gadyatskaya Aachen

   Leiden University

   Affiliation as printed

   Leiden University

5. Zilong Song

   Xi'an Jiaotong University

   Affiliation as printed

   Xi'an Jiaotong University

6. Yong Q. Cai

   California State University, Monterey Bay · IQVIA (United States)

   Affiliation as printed

   IQVIA Inc & California State University , Monterey Bay

7. Yang Liu

   University of California, Santa Cruz · Xi'an Jiaotong University

   Affiliation as printed

   University of California , Santa Cruz

   Xi'an Jiaotong University