Original title: ARIA: On the interaction between Architectures, Aggregation methods and Initializations in federated visual classification
Authors: Vasilis Siomos, Sergio Naval-Marimont, Jonathan Passerat-Palmbach, Giacomo Tarroni
In this article, the authors explore the concept of Federated Learning (FL), which is a collaborative training method that allows for privacy-preserving learning of models between different institutions. FL eliminates the need for sharing sensitive data by exchanging model parameters instead. While there have been individual studies on how client models are aggregated and the benefits of ImageNet pre-training, there is a lack of understanding on how the architecture chosen for FL affects the overall performance and how all the elements of FL are interconnected. To address this gap, the authors conduct a comprehensive study called ARchitecture-Initialization-Aggregation (ARIA) and benchmark its effectiveness on various medical image classification tasks. Their findings show that the best performance is achieved when ARIA elements are chosen together, rather than separately. The results also provide insights into good choices for each element depending on the task, the impact of normalization layers, and the usefulness of SSL pre-training. These findings have important implications for designing FL-specific architectures and training pipelines in the future.
Original article: https://arxiv.org/abs/2311.14625