Zeno: Byzantine-suspicious stochastic gradient descent
share
We propose Zeno, a new robust aggregation rule, for distributed synchronous Stochastic Gradient Descent (SGD) under a general Byzantine failure model. The key idea is to suspect the workers that are potentially malicious, and use a ranking-based preference mechanism. This allows us to generalize beyond past work--in our case, the number of malicious workers can be arbitrarily large, and we use only the weakest assumption on honest workers (at least one honest worker). We prove the convergence of SGD under these scenarios. Empirical results show that Zeno outperforms existing approaches under various attacks.
READ FULL TEXT