Federated Machine Learning Performance Evaluation with Flower: MNIST vs CIFAR-10 - Gandra. Shiva Krishna, Kota. Venkata Narayana, Jonnalagadda. Surya Kiran, Thatavarti. Satish, Inbarajan.P, Singanamalla. Jaya Mohnish (International Journal of Intelligent Systems and Applications in Engineering, 2024)
This paper delves into Federated Learning’s advantages, an innovative approach to distributed machine learning that enables model training using decentralized data, eliminating the necessity for centralized aggregation. The primary objective involves conducting extensive experiments with image datasets to discern the superior model training methodology between federated and non-federated approaches. Flower Federated Learning, an open-source framework integrated seamlessly with the TensorFlow machine learning platform, is utilized in the investigation. The pivotal point of this investigation revolves around a comparative analysis of federated and non-federated configurations, utilizing the renowned MNIST and CIFAR-10 datasets. Focus canters on key performance metrics such as precision, recall, and F1 score. In this paper, the experiment is conducted with a federated setup having two clients, each containing only 50% of random data from each dataset class. These clients are connected to a server where the global model is stored and updated as part of the experiment. The non-federated setup involves a single client for model training and testing. The federated setup achieves 92%-96% for MNIST and around 45%-50% for CIFAR-10 datasets over various rounds and 25 to 50 epochs. In contrast, the non-federated setup achieves around 96% for MNIST and 85% for CIFAR-10 datasets. These results represent a brief experimentation phase on a smaller client setup, with the potential for similar updates for a higher number of clients. However, it’s crucial to note that the field of federated learning is evolving rapidly, promising advancements that may narrow, if not bridge, the existing performance gap. The empirical analysis concludes that, as of now, traditional setups exhibit superior performance in comparison to federated configurations. Nevertheless, the evolving nature of federated learning warrants continued investigation and experimentation, as it holds significant potential to transform the landscape of distributed machine learning soon.