This is the code accompanying IEEE ISCC - DistInSys 2024 paper "FedDyS: Enhancing Federated Learning Efficiency with Dynamic Sample Selection " Paper link: []
FedDyS is a dynamic sample selection technique designed to enhance federated learning (FL) by reducing computational demands and mitigating data heterogeneity. It achieves this by eliminating non-essential training samples, which not only shrinks the training set size on local devices but also enhances data diversity. FedDyS preserves data privacy and prevents the catastrophic forgetting effect, common in FL, leading to improved accuracy and faster convergence. This makes FedDyS particularly suitable for low-resource settings, achieving optimal results with less than 15% of the usual number of training samples.
-
FedAvg -- Communication-Efficient Learning of Deep Networks from Decentralized Data (AISTATS'17)
-
FedProx -- Federated Optimization in Heterogeneous Networks (MLSys'20)
-
SCAFFOLD -- SCAFFOLD: Stochastic Controlled Averaging for Federated Learning (ICML'20)
-
MOON -- Model-Contrastive Federated Learning (CVPR'21)
-
FedSampling -- FedSampling: A Better Sampling Strategy for Federated Learning (JCAI-23)
Note that this code needs 3.10 <= python < 3.12.
To install necessary packages:
pip install -r requirements.txt
# partition the CIFAR-10 according to Dir(0.1) for 100 clients
cd data
python generate_data.py -d cinic10 -a 0.1 -cn 100
cd ../python3 -u ./src/server/fedDyS2.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "fedDyS2_CINIC10_transfer_th_mean" --th_mode mean --algo FedDyS
python3 -u ./src/server/fedDyS2.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "fedDyS2_CINIC10_transfer_th_1%" --th_mode percentage --th 0.01 --algo FedDyS
python3 -u ./src/server/fedDyS2.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "fedDyS2_CINIC10_transfer_th_5%" --th_mode percentage --th 0.05 --algo FedDyS
python3 -u ./src/server/fedDyS2.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "fedDyS2_CINIC10_transfer_th_15%" --th_mode percentage --th 0.15 --algo FedDyS
python3 -u ./src/server/fedDyS2.python3 -u ./src/server/fedavg.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "FedAVG_CINIC10_transfer" --algo FedAVG
python3 -u ./src/server/fedprox.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "FedProx_CINIC10_transfer" --algo FedProx
python3 -u ./src/server/moon.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "moon_CINIC10_transfer" --algo MOON
python3 -u ./src/server/fedsampling.py -d cinic10 -m mobile -jr 0.1 -ge 500 -le 5 --verbose_gap 1 --wandb 1 --run_name "fedsampling_CINIC10_transfer" --algo FedSampling| Argument | Description |
|---|---|
--dataset |
The name of dataset that experiment run on. |
--model |
The model backbone experiment used. |
--seed |
Random seed for running experiment. |
--join_ratio |
Ratio for (client each round) / (client num in total). |
--global_epoch |
Global epoch, also called communication round. |
--local_epoch |
Local epoch for client local training. |
--finetune_epoch |
Epoch for clients fine-tunning their models before test. |
--test_gap |
Interval round of performing test on clients. |
--eval_test |
Non-zero value for performing evaluation on joined clients' testset before and after local training. |
--eval_train |
Non-zero value for performing evaluation on joined clients' trainset before and after local training. |
--local_lr |
Learning rate for client local training. |
--momentum |
Momentum for client local opitimizer. |
--weight_decay |
Weight decay for client local optimizer. |
--verbose_gap |
Interval round of displaying clients training performance on terminal. |
--batch_size |
Data batch size for client local training. |
--use_cuda |
Non-zero value indicates that tensors are in gpu. |
--visible |
Non-zero value for using Visdom to monitor algorithm performance on localhost:8097. |
--global_testset |
Non-zero value for evaluating client models over the global testset before and after local training, instead of evaluating over clients own testset. The global testset is the union set of all client's testset. NOTE: Activating this setting will considerably slow down the entire training process, especially the dataset is big. |
--save_log |
Non-zero value for saving algorithm running log in FL-bench/out/${algo}. |
--straggler_ratio |
The ratio of stragglers (set in [0, 1]). Stragglers would not perform full-epoch local training as normal clients. Their local epoch would be randomly selected from range [--straggler_min_local_epoch, --local_epoch). |
--straggler_min_local_epoch |
The minimum value of local epoch for stragglers. |
--external_model_params_file |
The relative file path of external (pretrained) model parameters (*.pt). e.g., ../../out/FedAvg/mnist_100_lenet5.pt. Please confirm whether the shape of parameters compatible with the model by yourself. ⚠ This feature is enabled only when unique_model=False, which is pre-defined by each FL method. |
--save_model |
Non-zero value for saving output model(s) parameters in FL-bench/out/${algo}. The default file name pattern is ${dataset}_${global_epoch}_${model}.pt. |
--save_fig |
Non-zero value for saving the accuracy curves showed on Visdom into a .jpeg file at FL-bench/out/${algo}. |
--save_metrics |
Non-zero value for saving metrics stats into a .csv file at FL-bench/out/${algo}. |
--wandb |
whether to use Weights & Biases (W&B) |
--run_name |
the name of the experiment run for Weights & Biases (W&B) |
--algo |
algorithm name |
--th_mode |
threshold mode : mean or percentage |
--th |
threshold percentage |
This repository is derived from the FL-bench project: (for more details and to access more methods check this repository) https://github.com/KarhouTam/FL-bench/
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent