Results 1401 - 1410 of 24786
Federated learning must simultaneously handle statistical heterogeneity, adversarial participants, and privacy against a curious server. While each of these challenges is well understood in isolation, their interaction fundamentally changes what distributed learning algorithms can achieve.
I will present a tight characterization of the privacy–robustness–utility trade-off in distributed learning. We show that any algorithm that is both robust to a fraction of adversarial participants and locally differentially private must incur an unavoidable excess error. Beyond the separate costs of privacy and robustness, there is a coupling penalty governed by the corruption fraction and the privacy level. This phenomenon is orthogonal to statistical heterogeneity: even under homogeneous data, local privacy injects randomness that acts as artificial heterogeneity, which adversaries can exploit. I will conclude by discussing two structural responses to these limits: weakening the trust model through shared randomness, and reducing full collaboration via personalization.
This talk is based on the papers:
[AGS ICML’25] Towards Trustworthy Federated Learning with Untrusted Participants
[AGGPS ICML'23] On the Privacy-Robustness-Utility Trilemma in Distributed Learning
To improve trust and adoption of collaborative learning systems, it is important to provide opt-out guarantees which enable removal of unwanted (e.g. harmful or private) information after it has been used to train a model. A popular solution to this problem is approximate unlearning, which efficiently updates an LLM so that it behaves (roughly) as if it was not trained on a subset of data to begin with. However, existing methods are brittle in practice and can easily be attacked to reveal supposedly unlearned information. To alleviate issues with approximate unlearning, we instead propose SIFT-Masks (SIgn-Fixed Tuning-Masks), an exact unlearning method based on model merging. SIFT-Masks addresses two key limitations of standard model merging: (1) merging a large number of tasks can severely harm utility; and (2) methods that boost utility by sharing extra information across tasks make exact unlearning prohibitively expensive. SIFT-Masks solves these issues by (1) applying local masks to recover task-specific performance; and (2) constraining finetuning to align with a global sign vector as a lightweight approach to determine masks independently before merging. Across four settings where we merge up to 500 models, SIFT-Masks improves accuracy by 5-80% over naive merging and uses up to 250x less compute for exact unlearning compared to other merging baselines.