federated-learning-knowledge-distillation's Introduction

Federated-Learning-Knowledge-Distillation

repo in several methods FedAvg, FedMD, FedProto, FedProx, FedHKD.

main.py //For training the model
models.py //Our VAEs model for FMNIST, CIFAR10/100
sampling.py // functions that generate non-iid datasets for federated learning
utils.py // define functions that compute accuracy, soft prediction and model averaging
mem_utils.py // Library for monitoring memory usage and training time
option.py // define hyper-parameters
Server/*.py // object definition for server in each method
Client/*.py // object definition for client in each method

--dataset: 'CIFAR10', 'CIFAR100', ' SVHN'
--batch_size: 64 by defalut
--num_epochs: number of global rounds, 50 by defalut
--lr: learning rate, 0.001 by defalut
--lr_sh_rate: period of learning rate decay, 10 by defalut
--dropout_rate: drop out rate for each layer, 0.2 by defalut
--clip_grad: maximum norm for gradient
--num_users: number of clients, 10 by defalut
--sampling_rate: proportion of clients send updates per round, 1 by defalut
--local_ep: local epoch, 5 by defalut
--beta: concentration parameter for Dirichlet distribution: 0.5 by defalut
--seed: random seed(for better reproducting experiments): 0 by defalut
--std: standard deviation by Differential Noise, 2 by defalut
--code_len: length of latent vector, 32 by defalut
--alg: 'FedAvg, FedProx, Moon, FedMD, Fedproto, FedHKD'
--eval_only: only ouput the testing accuracy during training and the running time
--part: percentage of each local data
--temp: temperture for soft prediction
--lam: hyper-parameter for loss2
--gamma: hyper-parameter for loss3
--model: CNN resnet18 shufflenet
--save_model: save checkpoints of the model