A list of awesome papers and other cool resources on transfer learning and domain adaptation! Don't hesitate to suggest some resources I could have forgotten.
Papers are ordered by theme and inside each theme by publication date (submission date for arXiv papers). If the network or algorithm is given a name in a paper, this one is written in bold before the paper's name.
- A Survey on Transfer Learning (2009)
- A Survey of transfer learning (2016)
- Domain Adaptation for Visual Applications: A Comprehensive Survey (2017)
Only the source domain can have labels
- DANN: Domain-Adversarial Training of Neural Networks (2015)
- JAN: Deep Transfer Learning with Joint Adaptation Networks (2016)
- CoGAN: Coupled Generative Adversarial Networks (2016)
- DRCN: Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation (2016)
- DSN: Domain Separation Networks (2016)
- DIAT: Deep Identity-aware Transfer of Facial Attributes (2016)
- DTN: Unsupervised Cross-domain Image Generation (2016)
- Learning to Pivot with Adversarial Networks (2016)
- Pix2pix: Image-to-Image Translation with Conditional Adversarial Networks (2016)
- SimGAN: Learning from Simulated and Unsupervised Images through Adversarial Training (2016) (2016)
- PixelDA: Unsupervised Pixel–Level Domain Adaptation with Generative Adversarial Networks (2016)
- ADDA: Adaptative Discriminative Domain Adaptation (2017)
- UNIT: Unsupervised Image-to-Image Translation Networks (2017)
- CycleGAN: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (2017)
- DiscoGAN: Learning to Discover Cross-Domain Relations with Generative Adversarial Networks (2017)
- DualGAN: DualGAN: Unsupervised Dual Learning for Image-to-Image Translation (2017)
- GenToAdapt: Generate To Adapt: Aligning Domains using Generative Adversarial Networks (2017)
- SBADA-GAN: From source to target and back: symmetric bi-directional adaptive GAN (2017)
- WDGRL: Wasserstein Distance Guided Representation Learning for Domain Adaptation (2017)
- Model F: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos (2017)
- CyCADA: CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017)
- StarGAN: StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation (2017)
- pix2pixHD: High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
- XGAN: XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings (2017)
- Label Efficient Learning of Transferable Representations across Domains and Tasks (2017)
- DIRT-T: A DIRT-T Approach to Unsupervised Domain Adaptation (2017)
- ComboGAN: ComboGAN: Unrestrained Scalability for Image Domain Translation (2017)
- OT: Optimal Transport for Domain Adaptation (2015)
- Theoretical Analysis of Domain Adaptation with Optimal Transport (2016)
- JDOT: Joint distribution optimal transportation for domain adaptation (2017)
- Monge map learning: Large Scale Optimal Transport and Mapping Estimation (2017)
- DAassoc : Associative Domain Adaptation (2017)
- SurK: Covariate Shift in Hilbert Space: A Solution via Surrogate Kernels (2015)
- DAN: Learning Transferable Features with Deep Adaptation Networks (2015)
- RTN: Unsupervised Domain Adaptation with Residual Transfer Networks (2016)
- Easy DA: A Simple Approach for Unsupervised Domain Adaptation (2016)
- MCAE: Learning Classifiers from Synthetic Data Using a Multichannel Autoencoder (2015)
- SMCAE: Learning from Synthetic Data Using a Stacked Multichannel Autoencoder (2015)
- SGF: Domain Adaptation for Object Recognition: An Unsupervised Approach (2011)
- GFK: Geodesic Flow Kernel for Unsupervised Domain Adaptation (2012)
- SA: Unsupervised Visual Domain Adaptation Using Subspace Alignment (2015)
- CORAL: Return of Frustratingly Easy Domain Adaptation (2015)
- Deep CORAL: Deep CORAL: Correlation Alignment for Deep Domain Adaptation (2016)
- ILS: Learning an Invariant Hilbert Space for Domain Adaptation (2016)
- Log D-CORAL: Correlation Alignment by Riemannian Metric for Domain Adaptation (2017)
- MT: Self-ensembling for domain adaptation (2017)
- Adapting Visual Category Models to New Domains (2010)
- AdaBN: Revisiting Batch Normalization for Practical Domain Adaptation (2016)
All the source points are labelled, but only few target points are.
- da+lap-sim : Semi-Supervised Domain Adaptation with Instance Constraints (2013)
- EA++: Co-regularization Based Semi-supervised Domain Adaptation (2010)
- SDASL: Semi-supervised Domain Adaptation with Subspace Learning for Visual Recognition (2015)
Only a few target examples are available, but they are labelled
- FADA: Few-Shot Adversarial Domain Adaptation (2017)
- MNIST vs MNIST-M vs SVHN vs Synth vs USPS: digit images
- GTSRB vs Syn Signs : traffic sign recognition datasets, transfer between real and synthetic signs.
- NYU Depth Dataset V2: labeled paired images taken with two different cameras (normal and depth)
- CelebA: faces of celebrities, offering the possibility to perform gender or hair color translation for instance
- Office-Caltech dataset: images of office objects from 10 common categories shared by the Office-31 and Caltech-256 datasets. There are in total four domains: Amazon, Webcam, DSLR and Caltech.
- Cityscapes dataset: street scene photos (source) and their annoted version (target)
- UnityEyes vs MPIIGaze: simulated vs real gaze images (eyes)
- CycleGAN datasets: horse2zebra, apple2orange, cezanne2photo, monet2photo, ukiyoe2photo, vangogh2photo, summer2winter
- pix2pix dataset: edges2handbags, edges2shoes, facade, maps
- RaFD: facial images with 8 different emotions (anger, disgust, fear, happiness, sadness, surprise, contempt, and neutral). You can transfer a face from one emotion to another.
- VisDA 2017 classification dataset: 12 categories of object images in 2 domains: 3D-models and real images.
- Amazon review benchmark dataset: sentiment analysis for four kinds (domains) of reviews: books, DVDs, electronics, kitchen
- ECML/PKDD Spam Filtering: emails from 3 different inboxes, that can represent the 3 domains.
- 20 Newsgroup: collection of newsgroup documents across 6 top categories and 20 subcategories. Subcategories can play the role of the domains, as describe in this article.
The results are indicated as the prediction accuracy (in %) in the target domain after adapting the source to the target. For the moment, they only correspond to the results given in the original papers, so the methodology may vary between each paper and these results must be taken with a grain of salt.
Source Target |
MNIST MNIST-M |
Synth SVHN |
MNIST SVHN |
SVHN MNIST |
MNIST USPS |
USPS MNIST |
---|---|---|---|---|---|---|
SA | 56.90 | 86.44 | ? | 59.32 | ? | ? |
DANN | 76.66 | 91.09 | ? | 73.85 | ? | ? |
CoGAN | ? | ? | ? | ? | 91.2 | 89.1 |
DRCN | ? | ? | 40.05 | 81.97 | 91.80 | 73.67 |
DSN | 83.2 | 91.2 | ? | 82.7 | ? | ? |
DTN | ? | ? | ? | 90.66 | ? | ? |
PixelDA | 98.2 | ? | ? | ? | 95.9 | ? |
ADDA | ? | ? | ? | 76.0 | 89.4 | 90.1 |
UNIT | ? | ? | ? | 90.53 | 95.97 | 93.58 |
GenToAdapt | ? | ? | ? | 92.4 | 95.3 | 90.8 |
SBADA-GAN | 99.4 | ? | 61.1 | 76.1 | 97.6 | 95.0 |
DAassoc | 89.47 | 91.86 | ? | 97.60 | ? | ? |
CyCADA | ? | ? | ? | 88.3 | 94.8 | 95.7 |
No good library for the moment (as far as I know). If you're interested in a project of creating a generic transfer learning/domain adaptation library, please let me know.