Giter VIP home page Giter VIP logo

tf-differentiable-neural-computer's Introduction

Hybrid computing using a neural network with dynamic external memory

by Google DeepMind

Paper available here: http://www.nature.com/nature/journal/vaop/ncurrent/full/nature20101.html

This paper, published in Nature 2016, develops the idea of a differentiable neural computer (DNC). This paper is based heavily on the work Alex Graves and Greg Wayne previously did on "Neural Turing Machines" (link: https://arxiv.org/abs/1410.5401). I'd highly recommend that anybody attempting to really understand DNCs read and understand his paper first.

DNCs can be viewed as a more general type of LSTM (http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf) where the network learns how to use memory to understand data rather than attempting to learn the raw sequence relationships. This allows the network to be trained on a small amount of data and generalize to large amount of data without training, as well as handling inputs that were not necessarily seen during training --- a huge divergence from what was previously possible.

These findings are facilitated by the author's novel framework of memory which is fully differentiable (thus the name). Because of this property, the memory structure is able to be coupled with a neural network and trained by gradient descent or any other optimization method.

tf-differentiable-neural-computer's People

Contributors

claymcleod avatar tobiasweis avatar

Stargazers

Falguni Das Shuvo avatar Mingyuan Luo avatar Jan Bours avatar Denis Sidorenko avatar Arnon Bruno avatar  avatar  avatar Aruul Mozhi Varman Senthilkumar avatar  avatar José Olivares avatar SHRAVAN MURALI avatar B. Bueno avatar  avatar Michael Farnsworth avatar yuanke avatar Matt Shaffer avatar Thanachai Soontornwutikul avatar Florian Sestak avatar Aaron Chan avatar yosider avatar  avatar Sergey Zuev avatar Norman Heckscher avatar SErAphLi avatar Richard avatar  avatar gupengju avatar  avatar Anna avatar Nikita avatar zumpchke avatar Ignacio Cases avatar TENSORTALK avatar Joseph Cheng avatar nazoking avatar

Watchers

 avatar Matt Shaffer avatar  avatar  avatar Arnon Bruno avatar paper2code - bot avatar

tf-differentiable-neural-computer's Issues

I tiped make and get a error in the unzipped file

I tiped make and get a error in the unzipped file

joshua@joshua-HP-ENVY-m7-Notebook:/Downloads/tf-differentiable-neural-computer-master
$ make
python python/main.py copy pwd/data --summary-dir pwd/summary
Traceback (most recent call last):
File "python/main.py", line 4, in
from dnc import DNC
File "/home/joshua/Downloads/tf-differentiable-neural-computer-master/python/dnc.py", line 127
print("\rCompiling timestep {}/{} ({:.2f} %)".format(i+1, self.n_timesteps, ((i+1)/self.n_timesteps) * 100), end="")
^
SyntaxError: invalid syntax
Makefile:2: recipe for target 'run' failed
make: *** [run] Error 1
joshua@joshua-HP-ENVY-m7-Notebook:
/Down

which version of tensorflow it support?

I got error at tf1.2:

mldl@mldlUB1604:~/ub16_prj/tf-differentiable-neural-computer$ python3 python/main.py copy data --summary-dir summary
== COPY ==
[*] Loading from cached data...

== DNC ==

2017-08-02 03:47:45.524244: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 03:47:45.524267: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 03:47:45.524285: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 03:47:45.524288: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 03:47:45.524311: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 03:47:45.614177: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-08-02 03:47:45.614533: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties:
name: GeForce GTX 950M
major: 5 minor: 0 memoryClockRate (GHz) 1.124
pciBusID 0000:01:00.0
Total memory: 3.95GiB
Free memory: 3.66GiB
2017-08-02 03:47:45.614566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0
2017-08-02 03:47:45.614571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: Y
2017-08-02 03:47:45.614601: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 950M, pci bus id: 0000:01:00.0)
Compiling timestep 2/3 (66.67 %)Traceback (most recent call last):
File "python/main.py", line 35, in
optimizer="RMSProp")
File "/home/mldl/ub16_prj/tf-differentiable-neural-computer/python/dnc.py", line 57, in init
self.compile()
File "/home/mldl/ub16_prj/tf-differentiable-neural-computer/python/dnc.py", line 132, in compile
output, state = lstm_cell(input_, state)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 180, in call
return super(RNNCell, self).call(inputs, state)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/layers/base.py", line 441, in call
outputs = self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 383, in call
concat = _linear([inputs, h], 4 * self._num_units, True)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1017, in _linear
initializer=kernel_initializer)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 1065, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 962, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 360, in get_variable
validate_shape=validate_shape, use_resource=use_resource)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in _rnn_get_variable
variable = getter(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 352, in _true_getter
use_resource=use_resource)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 669, in _get_single_variable
found_var.get_shape()))
ValueError: Trying to share variable timestep_0/basic_lstm_cell/kernel, but specified shape (628, 2048) and found shape (568, 2048).
mldl@mldlUB1604:~/ub16_prj/tf-differentiable-neural-computer$

help on python folder input and output example

master/python$ python3 main.py
usage: main.py [-h] [--summary-dir SUMMARY_DIR] task datadir
main.py: error: the following arguments are required: task, datadir
joshua@joshua-HP-ENVY-m7-Notebook:~/Downloads/tf-differentiable-neural-computer-master/python$ python3 main.py -h
usage: main.py [-h] [--summary-dir SUMMARY_DIR] task datadir

positional arguments:
task
datadir Where do you want your datasets to be?

optional arguments:
-h, --help show this help message and exit
--summary-dir SUMMARY_DIR
Summary directory for tensorboard

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.