This is a caffe repository for transfer learning. We fork the repository with version ID 29cdee7 from Caffe and make our modifications. The main modifications are listed as follow:
- Add
mmd layerdescribed in paper "Learning Transferable Features with Deep Adaptation Networks". - Add
jmmd layerdescribed in paper "Deep Transfer Learning with Joint Adaptation Networks". - Add
entropy layerandouterproduct layerdescribed in paper "Unsupervised Domain Adaptation with Residual Transfer Networks". - Copy
grl layerandmessenger.hppfrom repository Caffe. - Emit
SOLVER_ITER_CHANGEmessage insolver.cppwheniter_changes.
If you have any problem about this code, feel free to concact us with the following email:
In data/office/*.txt, we give the lists of three domains in Office dataset.
We have published the Image-Clef dataset we use here.
In models/DAN/amazon_to_webcam, we give an example model based on Alexnet to show how to transfer from amazon to webcam. In this model, we insert mmd layers after fc7 and fc8 individually.
The bvlc_reference_caffenet is used as the pre-trained model. If the Office dataset and pre-trained caffemodel are prepared, the example can be run with the following command:
"./build/tools/caffe train -solver models/DAN/amazon_to_webcam/solver.prototxt -weights models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel"
Resnet pre-trainded model is here. We use Resnet-50.
In models/JAN, we give example models based on Alexnet and Resnet to show how to transfer from amazon to webcam, according to "Deep Transfer Learning with Joint Adaptation Networks". The shell scripts to train JAN model are the same with the above command, except that the paths of solver.prototxt and pretrained caffemodel are different.
As JAN uses the joint distribution of features (e.g. fc7) and softmax output (fc8) which are randomized at the beginning of training, one should stablize training by adding grl layer before JMMD Loss Layer to increase the loss weight from zero gradually, as suggested in the train_val.prototxt.
In mmd-layer and jmmd-layer, parameter loss_weight can be tuned to give mmd/jmmd loss different weights.
@inproceedings{DBLP:conf/icml/LongC0J15,
author = {Mingsheng Long and
Yue Cao and
Jianmin Wang and
Michael I. Jordan},
title = {Learning Transferable Features with Deep Adaptation Networks},
booktitle = {Proceedings of the 32nd International Conference on Machine Learning,
{ICML} 2015, Lille, France, 6-11 July 2015},
pages = {97--105},
year = {2015},
crossref = {DBLP:conf/icml/2015},
url = {http://jmlr.org/proceedings/papers/v37/long15.html},
timestamp = {Tue, 12 Jul 2016 21:51:15 +0200},
biburl = {http://dblp2.uni-trier.de/rec/bib/conf/icml/LongC0J15},
bibsource = {dblp computer science bibliography, http://dblp.org}
}
@inproceedings{DBLP:conf/nips/LongZ0J16,
author = {Mingsheng Long and
Han Zhu and
Jianmin Wang and
Michael I. Jordan},
title = {Unsupervised Domain Adaptation with Residual Transfer Networks},
booktitle = {Advances in Neural Information Processing Systems 29: Annual Conference
on Neural Information Processing Systems 2016, December 5-10, 2016,
Barcelona, Spain},
pages = {136--144},
year = {2016},
crossref = {DBLP:conf/nips/2016},
url = {http://papers.nips.cc/paper/6110-unsupervised-domain-adaptation-with-residual-transfer-networks},
timestamp = {Fri, 16 Dec 2016 19:45:58 +0100},
biburl = {http://dblp.uni-trier.de/rec/bib/conf/nips/LongZ0J16},
bibsource = {dblp computer science bibliography, http://dblp.org}
}
@inproceedings{DBLP:conf/icml/LongZ0J17,
author = {Mingsheng Long and
Han Zhu and
Jianmin Wang and
Michael I. Jordan},
title = {Deep Transfer Learning with Joint Adaptation Networks},
booktitle = {Proceedings of the 34th International Conference on Machine Learning,
{ICML} 2017, Sydney, NSW, Australia, 6-11 August 2017},
pages = {2208--2217},
year = {2017},
crossref = {DBLP:conf/icml/2017},
url = {http://proceedings.mlr.press/v70/long17a.html},
timestamp = {Tue, 25 Jul 2017 17:27:57 +0200},
biburl = {http://dblp.uni-trier.de/rec/bib/conf/icml/LongZ0J17},
bibsource = {dblp computer science bibliography, http://dblp.org}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent