FedCova: Robust Federated Covariance Learning Against Noisy Labels

AI 总结

• 针对联邦学习中噪声标签导致的局部过拟合与全局性能下降，提出依赖最小化的FedCova方法。
• 以互信息最大化为基础，设计仅依赖类特征协方差并包含误差容忍项的联邦有损特征编码目标。
• 利用协方差表征的特征子空间，构建子空间增强分类器，并统一特征学习、分类器构造与噪声标签纠正三过程。
• 在对称与非对称噪声、异构数据分布下进行验证，并在CIFAR-10/100与Clothing1M上优于现有SOTA方法。

内容摘录

Noisy labels in distributed datasets induce severe local overfitting and consequently compromise the global model in federated learning (FL). Most existing solutions rely on selecting clean devices or aligning with public clean datasets, rather than endowing the model itself with robustness. In this paper, we propose FedCova, a dependency-free federated covariance learning framework that eliminates such external reliances by enhancing the model's intrinsic robustness via a new perspective on feature covariances. Specifically, FedCova encodes data into a discriminative but resilient feature space to tolerate label noise. Built on mutual information maximization, we design a novel objective for federated lossy feature encoding that relies solely on class feature covariances with an error tolerance term. Leveraging feature subspaces characterized by covariances, we construct a subspace-augmented federated classifier. FedCova unifies three key processes through the covariance: (1) training the network for feature encoding, (2) constructing a classifier directly from the learned features, and (3) correcting noisy labels based on feature subspaces. We implement FedCova across both symmetric and asymmetric noisy settings under heterogeneous data distribution. Experimental results on CIFAR-10/100 and real-world noisy dataset Clothing1M demonstrate the superior robustness of FedCova compared with the state-of-the-art methods.