Unofficial EfficientNetV2 pytorch implementation repository.
It contains:
- Simple Implementation of model (here)
- Pretrained Model (numpy weight, we upload numpy files converted from official tensorflow checkout point)
- Training code (here)
- Tutorial (Colab EfficientNetV2-predict tutorial, Colab EfficientNetV2-finetuning tutorial)
- Experiment results
- Tutorial
- Experiment results
- Experiment Setup
- References
Colab Tutorial
-
How to use model on colab? please check Colab EfficientNetV2-predict tutorial
-
How to train model on colab? please check Colab EfficientNetV2-finetuning tutorial
-
See how cutmix, cutout, mixup works in Colab Data augmentation tutorial
If you just want to use pretrained model, load model by torch.hub.load
import torch
model = torch.hub.load('hankyul2/EfficientNetV2-pytorch', 'efficientnet_v2_s', pretrained=True, nclass=1000)
print(model)
Available Model Names: efficientnet_v2_{s|m|l}
(ImageNet), efficientnet_v2_{s|m|l}_in21k
(ImageNet21k)
If you want to finetuning on cifar, use this repository.
-
Clone this repo and install dependency
git clone https://github.com/hankyul2/EfficientNetV2-pytorch.git pip3 install requirements.txt
-
Train & Test model
python3 main.py fit --config config/base.yaml --trainer.gpus 2, --data.dataset_name cifar100 --model.model_name efficientnet_v2_s --seed_everything 2021
Model Name | Pretrained Dataset | Cifar10 | Cifar100 |
---|---|---|---|
EfficientNetV2-S | ImageNet | 97.98 | 88.53 |
EfficientNetV2-M | ImageNet | 98.38 | 85.81 (๐ค) |
EfficientNetV2-L | ImageNet | 98.4 | - |
EfficientNetV2-S-in21k | ImageNet21k | 98.1 | 89.2 |
EfficientNetV2-M-in21k | ImageNet21k | 98.2 | 89.5 |
EfficientNetV2-L-in21k | ImageNet21k | 98.2 | 90.1 |
EfficientNetV2-XL-in21k | ImageNet21k | - | - |
Note
- Training Results are not good enough to match with paper results
- All models are trained using same setup in experiment setup section (which is adapted from paper)
ImageNet Setup
Setup | Contents |
---|---|
Data | ImageNet(ImgeSize=128, RandAugmentation=5, Mixup=0) |
Model | EfficientNetV2(Dropout=0.1, Stochastic_depth=0.2) |
Optimizer | RMSProp(decay=0.9, batch_norm_momentum=0.99, weight_decay=1e-5, momentum=0.9) |
Learning rate | (epoch=350, batch_size=4096, lr=0.256, warmup=?) learning rate decay by 0.97 every 2.4 epochs |
EMA | decay_rate=0.9999 |
Cifar Setup
Setup | Contents |
---|---|
Data | Cifar(ImgeSize=224, Cutmix) |
Model | EfficientNetV2(Dropout=0.0, Stochastic_depth=0.2) |
Optimizer | SGD(weight_decay=1e-5, momentum=True) |
Learning rate | CosineLearningRate(epoch=100, batch_size=32, lr=0.001, warmup=1) |
Note
- For progressive learning,
ImageSize
,RandAugmentation
,Mixup
,Dropout
are going to be changed along with epoch. - Evaluation Size is different for each model
epoch=100
in Cifar Stepup is calculated from paper like this:10,000 step * 512 batch size / 50,000 images = 102.4
- To see more model specific details, check efficientnet_v2_config.py
- To see more train hyperparameter, check cifar.yaml
EfficientNetV2
-
Title: EfficientNetV2: Smaller models and Faster Training
-
Author: Minxing Tan
-
Publication: ICML, 2021
-
Link: Paper | official tensorflow repo | other pytorch repo
-
Other references: