Towards Efficient, Robust, and Adaptive Distributed Learning

Distributed learning has become a critical research area due to the rapid growth of data and the demand for scalable Machine Learning (ML). However, ensuring efficient and robust training remains challenging in large-scale systems where data and computation are spread across numerous machines. In this talk, I will present three works that address several of these challenges.

First, I propose an efficient first-order method for stochastic optimization with data originating from a Markov chain—a fundamental problem in peer-to-peer decentralized optimization. This method achieves near-optimal convergence rates while eliminating the need for prior knowledge of the mixing time typically required in previous work. Second, I introduce a downlink compression framework for cross-device Federated Learning, where low-resource clients may participate only intermittently and lack persistent memory, that significantly reduces bandwidth consumption while matching the accuracy of an uncompressed baseline model. Finally, I address fault-tolerance in distributed ML (aka Byzantine-robust learning) by developing an algorithm that adapts to dynamic adversaries whose behavior might change over time—unlike previous approaches that assume a fixed set of Byzantine workers—without requiring knowledge of the fraction of malicious machines.  

Ph.D. student under the supervision of Prof. Kfir Y. Levy.