Performance Analysis on Federated Learning with Differential Privacy
share
In this paper, to effectively prevent the differential attack, we propose a novel Federated learning (FL) framework based on the concept of differential privacy (DP), in which artificial noises are added to the parameters at the clients side before being uploaded for aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting the variances of artificial noises. Then we develop a theoretical convergence bound of the loss function of the trained FL model in the NbAFL. Our developed bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i.e., a better convergence performance leads to a lower protection level; 2) Increasing the number N of overall clients participating in FL can improve the convergence performance, given a fixed privacy protection level; 3) There is an optimal number of maximum aggregation times in terms of convergence performance for a given protection level. Furthermore, we propose a K-random scheduling strategy, where K (1<K<N) clients are randomly selected from the N overall clients to participate in each aggregation. We also develop the corresponding convergence bound of the loss function in this case. From our analysis, the K-random scheduling strategy can retain the above three properties. More interestingly, we find that there is an optimal K that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the designs on various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.
READ FULL TEXT