Official Fast AutoAugment implementation in PyTorch.

• Fast AutoAugment learns augmentation policies using a more efficient search strategy based on density matching.
• Fast AutoAugment speeds up the search time by orders of magnitude while maintaining the comparable performances.

We do not open augmentation search codes at this moment, but it will be publicly open with our follow-up studies.

Search : 3.5 GPU Hours (1428x faster than AutoAugment), WResNet-40x2 on Reduced CIFAR-10

Search : 450 GPU Hours (33x faster than AutoAugment), ResNet-50 on Reduced ImageNet

You can train network architectures on CIFAR-10 / 100 and ImageNet with our searched policies.

• fa_reduced_cifar10 : reduced CIFAR-10(4k images), WResNet-40x2
• fa_reduced_imagenet : reduced imagenet(50k images, 120 classes), ResNet-50

$ python train.py -c confs/wresnet40x2_cifar10_b512.yaml --aug fa_reduced_cifar10 --dataset cifar10
$ python train.py -c confs/wresnet40x2_cifar10_b512.yaml --aug fa_reduced_cifar10 --dataset cifar100
$ python train.py -c confs/wresnet28x10_cifar10_b512.yaml --aug fa_reduced_cifar10 --dataset cifar10
$ python train.py -c confs/wresnet28x10_cifar10_b512.yaml --aug fa_reduced_cifar10 --dataset cifar100

Note that we conducted experiments with ImageNet dataset using 8 machines with four V100 GPUs each.

$ python train.py -c confs/resnet50_b4096.yaml --aug fa_reduced_imagenet --horovod

If you use any part of this code in your research, please cite our paper.

@article{lim2019fast,
  title={Fast AutoAugment},
  author={Lim, Sungbin and Kim, Ildoo and Kim, Taesup and Kim, Chiheon and Kim, Sungwoong},
  journal={ICML AutoML workshop},
  year={2019}
}

• Ildoo Kim, [email protected]
• Sungbin Lim, [email protected]

1. ResNet References
   • (ResNet) Deep Residual Learning for Image Recognition
     • Paper : https://arxiv.org/abs/1512.03385
   • (ResNet) Identity Mappings in Deep Residual Networks
     • Paper : https://arxiv.org/abs/1603.05027
   • Codes
     • https://github.com/osmr/imgclsmob/tree/master/pytorch/pytorchcv/models
2. (PyramidNet) Deep Pyramidal Residual Networks
   • Paper : https://arxiv.org/abs/1610.02915
   • Author's Code : https://github.com/dyhan0920/PyramidNet-PyTorch
3. (Wide-ResNet)
   • Code : https://github.com/meliketoy/wide-resnet.pytorch
4. (Shake-Shake)
   • Code : https://github.com/owruby/shake-shake_pytorch
5. ShakeDrop Regularization for Deep Residual Learning
   • Paper : https://arxiv.org/abs/1802.02375
   • Code : https://github.com/owruby/shake-drop_pytorch
6. LARS : Large Batch Training of Convolutional Networks
   • Paper : https://arxiv.org/abs/1708.03888
   • Code : https://github.com/noahgolmant/pytorch-lars/blob/master/lars.py
7. (ARS-Aug) Learning data augmentation policies using augmented random search
   • Paper : https://arxiv.org/abs/1811.04768
   • Author's Code : https://github.com/gmy2013/ARS-Aug
8. AutoAugment
   • Code : https://github.com/tensorflow/models/tree/master/research/autoaugment
9. https://pytorch.org/docs/stable/torchvision/models.html
10. https://github.com/eladhoffer/convNet.pytorch/blob/master/preprocess.py
11. Ray
    • https://github.com/ray-project/ray
    • https://ray.readthedocs.io/en/latest/tune.html
    • https://medium.com/formcept/scaling-python-modules-using-ray-framework-e5fc5430dc3e
12. HyperOpt
    • https://github.com/hyperopt/hyperopt