Semi-supervised-Federated-Learning-with-non-IID-Data-Algorithm-and-System-Design (HPCC'21)

Author: Zhe Zhang, Shiyao Ma, Zehui Xiong, Yi Wu, Qiang Yan, Xiaoke Xu, Dusit Niyato, Fellow, IEEE

Abstract

In this paper, we design a Semi-supervised Federated Learning(SSFL) framework to slove classification problem.

---

Dataset

We use the CIFAR-10 dataset including 56,000 training samples and 2,000 test samples as the validation dataset in our experiment.

We also use the Fashion-MNIST dataset including 64,000 training samples and 2,000 test samples as the validation dataset.

Furthermore, we introduce Dirchlet distribution function to simulate the different non-IID level scenario in our experiment. We control Dirchlet distribution via modify parameters in /modules/data_generator.py.

z = np.random.dirichlet((0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1), size=10)

---

Framework

In this paper, we design a robust SSFL framework that uses the proposed FedMix algorithm to achieve high-precision semi-supervised learning.

And we make some improvements on the code of this paper and you also can cite this paper:

@inproceedings{jeong2020federated,
  title={Federated Semi-Supervised Learning with Inter-Client Consistency \& Disjoint Learning},
  author={Jeong, Wonyong and Yoon, Jaehong and Yang, Eunho and Hwang, Sung Ju},
  booktitle={International Conference on Learning Representations},
  year={2020}
}

---

How to run

• step 1: Download the dataset and prepare the IID or Non-IID data via the following command lines:

python3 main.py -j data -t ls-biid-c10
python3 main.py -j data -t lc-bimb-c10

• step 2: If you want to run FedMix after generate the IID or Non-IID data, you can use the following command lines:

python main.py -g 0,1 -t ls-bimb-c10 -f 0.05
python main.py -g 0,1 -t ls-biid-c10 -f 0.05

---

Experimental results

Test accuracy curves on IID

Test accuracy curves on Non-IID

Citation

Coming soon...