albanie / pytorch-mcn Goto Github PK

View Code? Open in Web Editor NEW

93.0 4.0 13.0 176 KB

Convert models from MatConvNet to PyTorch

License: MIT License

Python 79.56% MATLAB 14.10% Shell 6.34%

pytorch-mcn's Introduction

pytorch-mcn

A tool for importing trained MatConvNet models into Pytorch (if you were hoping to go the other way, try mcnPyTorch).

Demo

To run the importer, set the path to the MatConvNet model and supply an output directory (where the imported PyTorch models will be stored) in the importer.sh script. There are a few examples in the script which can be commented/uncommented to run as a demo. Then run bash importer.sh.

Imported Models

A number of standard models have been imported and verified, and can be found here.

Verification

Verifying an imported model requires MATLAB and a copy of MatConvNet (the specific dependencies are given in compare/startup.m). The process is as follows:

Run the compare/featureDumper.m script to dump intermediate features from the original MatConvNet model to disk.
Import model to PyTorch in debug_mode (an option that can be set in importer.sh. This will generate additional source code in the PyTorch model definition that stores every intermediate tensor computed by the network.
Run the compare/compare_models.py script, which will perform a numerical comparison between the tensors.

Notes

Model conversion between frameworks can be challenging, because the layers and modules of each framework do not have an exact correspondence. As a result, there is often a bit of work involved in the conversion process, particularly for non-standard architectures.

This tool requires at least Python 3.5 and PyTorch 0.3.0 (by default, ipython will be used, but you can switch to standard python by changing a config variable in importer.sh). The tool expects the MatConvNet models to be in dagnn format (the ensure_dags.m script converts models to this format from SimpleNN if required).

Ideally in future the model conversion process will run via onnx but it seems that currently quite a lot of support is missing for required functionality. The plan is therefore to update the converter when possible.

pytorch-mcn's People

Contributors

Stargazers

Watchers

Forkers

shubhampachori12110095 sandbox3aster lifeng9472 cuiwenxue naydyonov tinyloop kezhende kartheekmedathati matkowski-voy wang93 rsrscoder monurcan sybil-li-x

pytorch-mcn's Issues

for key in ['met','params','layers']: assert key in mcn.keys()

I am new in DeepLearning and trying to convert a pre-trained net from matconvnet to pytorch.
The matconvnet version is 1.0-beta13, i re-saved it in matlab with -v7.
When i run the script it says:
"for key in ['meta','params','layers']: assert key in mcn.keys()" in "...../pytorch-mcn/python/importer.py", line 49, in load_mcn_net

Can anyone teach how to use the lib?
The list of example net is a list of complicated models like "vgg,Alex..." ,but i only need to convert a simple net with only conv and relu and concat.
I download the pre-trained model here
https://github.com/Yijunmaverick/DeepJointFilter/tree/master/examples/Test/Upsampling_8x.mat

SyntaxError: invalid syntax in importer.pyline 277, can you help me?

When I imports the trained MatConvNet model https://pan.baidu.com/s/1kWpxAtT into Pytorch, which is trained with code https://github.com/layumi/Pedestrian_Alignment , it get the error as follow: can you import it, verify and public it on your web?
importing net-epoch-23
flattening at
Exporting MatConvNet model to PyTorch (may take some time)...
File "/home/chengzi/pytorch-mcn/python/importer.py", line 277
arch = sg.build_header_str(self.name, **self.meta, debug_mode=self.debug_mode)
^
SyntaxError: invalid syntax

How to use the model?

I have download the "imagenet_mcn_vgg_m (weights)", how can I use it in pytorch? And, what the mean and var is of the input image?

vgg-m-2048 without BatchNorm model

It seems that, the magenet_mcn_vgg_m in your pre-convert model is the vgg-m with BN. Now, the pytorch supply the LRN, could you please supply the pre-convert model of vgg-m-2048 without BatchNorm but with LRN?

SyntaxError: invalid syntax

*** SyntaxError: invalid syntax
> /home/gece/Workspace/pytorch-mcn/python/importer.py(239)extract_dag()
    238         else: import ipdb ; ipdb.set_trace()
--> 239         for output, out_ch in zip(node['outputs'], out_chs):
    240             in_ch_store[output] = out_ch

ipdb>

I got the above error when trying to convert imagenet-vgg-f and imagenet-vgg-m-1024. I have already converted them to dags using ensure_dags.m.
( I use python3.5, and I can convert imagenet-vgg-verydeep-16 successfully )

Could not download the pre-trianed models now

Dear Professor,

I could not download the pre-trianed models now.
Could you update the links?
I need the model named vgg_face_dag.pth.

Thanks,
Chawdoe

Loading a network with empty 'meta'

I am trying to load the ICLabel model in PyTorch, named netICL.mat. Opening the model in Matlab, I can see that the argument meta is empty.

Thus, the converter is failing in build_network on line 537:

normalization = mcn_net['meta']['normalization']

How can I solve this?

I suggest change python/ipython to python3/ipython3 for using Python3.5 obviously in importer.sh

Steps for Convert models from MatConvNet to PyTorch with pytorch-mcn
(1)run pytorch-mcn/compare/ensure_dags.m in matlab to convert models to dagnn...
(2)edit and run shell script importer.sh to Convert models from MatConvNet to PyTorch with pytorch-mcn
(3)the results
Saving imported model definition to ./models/resnext_50_32x4d_pt_mcn.py
Saving imported weights to ./models/resnext_50_32x4d_pt_mcn.pth

Bugs?
(1)Due to the default Python is 2.7 on the most Linux OS, and Python3 means Python3.5 on the the most Linux OS. Thus, I suggest change the python/ipython to python3/ipython3 for using Python3.5 obviously in importer.sh.
(2) The waning "skipping conv bias term"
skipping conv bias term
skipping conv bias term
skipping conv bias term
skipping conv bias term
skipping conv bias term
(3) Errors
> /home/chengzi/Downloads/pytorch-mcn/python/ptmcn_utils.py(66)conv2d_mod()
65 if not valid_conv_groups: import ipdb ; ipdb.set_trace()
---> 66 assert valid_conv_groups, msg
67 num_groups = int(in_ch / fsize[2])