• We propose a method of learning in the frequency domain (using DCT coefficients as input), which requires little modification to the existing CNN models that take RGB input.
• We show that learning in the frequency domain better preserves image information in the pre-processing stage than the conventional spatial downsampling approach.
• We propose a learning-based dynamic channel selection method to identify the trivial frequency components for static removal during inference. Experiment results on ResNet-50 show that one can prune up to $87.5%$ of the frequency channels using the proposed channel selection method with no or little accuracy degradation in the ImageNet classification task.
• To the best of our knowledge, this is the first work that explores learning in the frequency domain for high-level vision tasks, such as object detection and instance segmentation.

Please refer to the image classfication and instance segmentation sections for more details.

If you use our code/models in your research, please cite our paper:

@InProceedings{Xu_2020_CVPR,
  author = {Xu, Kai and Qin, Minghai and Sun, Fei and Wang, Yuhao and Chen, Yen-Kuang and Ren, Fengbo},
  title = {Learning in the Frequency Domain},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2020}
}