zer0n / deepframeworks Goto Github PK

Hi zer0n,
Can you add MXNet、Petuum、IDLF、Neon to the list and compare them.

Thanks.

hey, one does not need to implement full forward/backward for every new layer in torch, layers can be composed just like in Theano.

For example, LSTM:
https://github.com/karpathy/char-rnn/blob/master/model/LSTM.lua

GRU:
https://github.com/karpathy/char-rnn/blob/master/model/GRU.lua

Well written article!

I would differ on the Model Deployment stars for Torch though : Torch runs on LuaJIT, and LuaJIT runs on a shocking number of platforms: x86, ARM, PS3, PS4, XBox, PPC, MIPS, Android, iOS, embedded devices.

I would suggest boosting the stars to reflect this.

• What optimization methods are supported
• Their performances

Torch recently launched pytorch that supports python. Would love a revisit :)

CNTK is slightly worse than Caffe because there's no Python or any other high-level language interface

There is now: https://github.com/Microsoft/CNTK/tree/master/contrib/Python

Is there this type of deep learning model?
There are two labeled folders for binary classification.
ex) men and women, cats oand dogs, etc.
And then inserting the images to each folders as training data.
And then just run a simple command to train.
That’s all.
I need these simple training network model. Is there any?

Thanks for creating this comparison page. I think it will be usefull for many people.
Few comments:

1. CNTK Multi-GPU. The paper you mentioned only presents results for fully-connected networks. And it can not be compared to distributed training of CNNs, for example. It turns out that distributed training of CNNs is much harder in a sense of acheiving high scalability factor. Currently noone demonstrated predictable and fast distributed training of CNNs.
2. Model deployment. I think Torch mark could be 0.5 higher. The reason is since it is written on C and Lua, and lua in turn also written on C, very compact and embeddable, you actually can make Torch models run on exotic processors or DSPs (and even FPGA) which only have C compiler available.
3. Architecture. This might be subjective but I would give Torch smaller mark. nn module alone looks good, but Torch is more then just nn, so as an inheritance of using Lua you have to use a lot of other modules.

https://docs.chainer.org/en/stable/comparison.html

Recently a couple of bindings have been released for other languages for TensorFlow, these include Haskell, Rust and Ruby:
https://github.com/tensorflow/haskell
https://github.com/tensorflow/rust
https://github.com/somaticio/tensorflow.rb

This would affect the 'interfaces' section here:
https://github.com/zer0n/deepframeworks#interfaces

Torch does have a computational graph approach available via nngraph, or Autograd. On the flip side, Lasagne was created to add layers for Theano. I also prefer the computational graph as an abstraction for neural networks, but clearly both sides have their pros and cons. It should be worth mentioning this, especially considering a merge of nn and nngraph is on the roadmap.

On the other weakness, it's not part of the distributed packages, but there is also a Python interface of sorts. If you can at least write a #subjective tag that would be good - programming languages and the ecosystems that come with them are subject to a wide range of opinions.

You mentioned:

The lack of static type in Python is just ... painful :).

However, now we have mypy and accompanying PEP 484: Type Hints, along with WIP PEP 526: Syntax for Variable and Attribute Annotations.

Very good overview, but I think you missed a feature in caffe - Networks can be written in python in recent versions of caffe. See for example:
https://github.com/beijbom/beijbom_python_lib/blob/master/beijbom_caffe_tools.py
There is a python syntax to generate the network, which is then written to text file and re-read upon training.

Hi zer0n
Can you add Keras to the list and compare them.
Thanks!

Hi there. Well written, just wanted to add that even though there isn't any official support for tensorflow under windows, I can confirm it does run under the bash for ubuntu for windows beta. Haven't tried GPU though.

Great writing!

Could you add whether a package supports (Or can Interface) with MATLAB?
Moreover, does it support Windows or Linux only?

Thank You.

Great work, thanks!
It clarified me a lot of things.

I would love to see also DL4J in this comparison.
Also capacity to train and run models on a cluster may be a key factor for many.

With gold stars it will look better

Caffe doesn't require x86 or CUDA and has already been ported to iOS and Android.

You've done a great review of deep learning toolkits and I thank you for that. However, they all are a pain to interop from Java world. Have you stumbled upon such a framework that would compare well to them and would be closer to the JVM? I can think of deeplearning4j but I'm not sure it fits the kind of evaluation you are doing.

I disagree with your review of Torch on Module deployement. Lua and Torch are easily deployed on smart phones running iOS and Android. As for server to server, Lua can be interfaced with C, C++, CUDA, Java, Python and many other languages. This makes it easy to integrate into existing systems, i.e. to deploy. I agree that it has poor visualisation libraries. So 4 and 1/2 stars instead of 3?

Please see the below linked chart for details on my claims.

https://github.com/Microsoft/CNTK/blob/master/Documentation/Documents/PerformanceChart.png

Thanks.

Looks like a typo.