bgavran / dnc Goto Github PK

Would you consider adding a licence for this? Also, are you interested in contributions to the repo?

Summary generated. Step 74400 Test cost == 31.212728500 Time == 8.57s
Summary generated. Step 74500 Test cost == 10.503337860 Time == 8.38s
Summary generated. Step 74600 Test cost == 749542.875000000 Time == 8.54s
Summary generated. Step 74700 Test cost == 866.475341797 Time == 8.72s
Summary generated. Step 74800 Test cost == 6.063397408 Time == 8.58s
Summary generated. Step 74900 Test cost == 19.894033432 Time == 8.50s
Summary generated. Step 75000 Test cost == 3009.345458984 Time == 8.53s
Model saved!
Summary generated. Step 75100 Test cost == 401.853302002 Time == 8.68s
Summary generated. Step 75200 Test cost == 201.970886230 Time == 8.54s
Summary generated. Step 75300 Test cost == 4504.419433594 Time == 8.59s
Summary generated. Step 75400 Test cost == 6352.577148438 Time == 8.56s
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1139, in _do_call
return fn(*args)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1121, in _run_fn
status, run_metadata)
File "/usr/lib/python3.5/contextlib.py", line 66, in exit
next(self.gen)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: DNC/FF_step/dense_1/bias_0
[[Node: DNC/FF_step/dense_1/bias_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](DNC/FF_step/dense_1/bias_0/tag, DNC/FF_step/dense_1/bias/read)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 54, in
dnc.run_session(task, Hp, project_path) # , restore_path=restore_path)
File "/home/mldl/ub16_prj/DNC/src/controller.py", line 94, in run_session
mask: m})
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: DNC/FF_step/dense_1/bias_0
[[Node: DNC/FF_step/dense_1/bias_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](DNC/FF_step/dense_1/bias_0/tag, DNC/FF_step/dense_1/bias/read)]]

Caused by op 'DNC/FF_step/dense_1/bias_0', defined at:
File "main.py", line 54, in
dnc.run_session(task, Hp, project_path) # , restore_path=restore_path)
File "/home/mldl/ub16_prj/DNC/src/controller.py", line 57, in run_session
tf.summary.histogram(variable.name, variable)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/summary/summary.py", line 221, in histogram
tag=scope.rstrip('/'), values=values, name=scope)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 131, in _histogram_summary
name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Nan in summary histogram for: DNC/FF_step/dense_1/bias_0
[[Node: DNC/FF_step/dense_1/bias_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](DNC/FF_step/dense_1/bias_0/tag, DNC/FF_step/dense_1/bias/read)]]

mldl@mldlUB1604:/ub16_prj/DNC/src$
mldl@mldlUB1604:/ub16_prj/DNC/src$
mldl@mldlUB1604:~/ub16_prj/DNC/src$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
tf.version
'1.2.1'

Hello, I want to compare the lstm and the dnc results as you did. However, when I run your code main.py in the memory part, I have the following problem :

File "/Users/pauldaoudi1/Downloads/DNC-master-2/src/controller_implementations/dnc/memory.py", line 92, in step
interface_vector = controller_output @ self.interface_weights # shape [batch_size, interf_vector_size]

TypeError: unsupported operand type(s) for @: 'Tensor' and 'Variable'.

Do you have an idea where the error could be ?
Thank you very much.

Paul.

python3 main.py
Loaded task
Loaded controller
WARNING:tensorflow:From /home/joshua/Downloads/DNC-master/src/controller_implementations/dnc/memory.py:228: calling softmax (from tensorflow.python.ops.nn_ops) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
WARNING:tensorflow:From /home/joshua/Downloads/DNC-master/src/controller_implementations/dnc/memory.py:215: calling l2_normalize (from tensorflow.python.ops.nn_impl) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
This model has 159800 parameters!
2018-03-10 09:46:22.976608: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-03-10 09:46:23.107730: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-10 09:46:23.108172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: GeForce 940M major: 5 minor: 0 memoryClockRate(GHz): 1.176
pciBusID: 0000:04:00.0
totalMemory: 1.96GiB freeMemory: 1.67GiB
2018-03-10 09:46:23.108200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-03-10 09:46:23.691006: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1426 MB memory) -> physical GPU (device: 0, name: GeForce 940M, pci bus id: 0000:04:00.0, compute capability: 5.0)
Starting...
Summary generated. Step 0 Test cost == 0.696296692 Time == 2.56s
Summary generated. Step 100 Test cost == 0.551536858 Time == 6.58s
Summary generated. Step 200 Test cost == 0.290066570 Time == 7.59s
Summary generated. Step 300 Test cost == 0.282800913 Time == 7.60s

How do you run tensorboard on the DNC program? it asks for arguments when I run tensorboard when main.py is running.

So this is what I see in your paper I go to the command line and hit tensorboard

To start the DNC with feedforward controller on the copy task with curriculum learning just run the main.py file. It should work out of the box and all the memory operations will be automatically visualized in Tensorboard.

Traceback (most recent call last):
File "/usr/local/bin/tensorboard", line 11, in
sys.exit(run_main())
File "/home/joshua/.local/lib/python3.6/site-packages/tensorboard/main.py", line 36, in run_main
tf.app.run(main)
File "/home/joshua/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "/home/joshua/.local/lib/python3.6/site-packages/tensorboard/main.py", line 45, in main
default.get_assets_zip_provider())
File "/home/joshua/.local/lib/python3.6/site-packages/tensorboard/program.py", line 165, in main
tb = create_tb_app(plugins, assets_zip_provider)
File "/home/joshua/.local/lib/python3.6/site-packages/tensorboard/program.py", line 189, in create_tb_app
raise ValueError('A logdir must be specified when db is not specified. '
ValueError: A logdir must be specified when db is not specified. Run tensorboard --help for details and examples.

a step by step to use csv inputs and get csv or screen outputs would be helpful I do not know how to do that
how to get information from a python program that gets vareables from a time series moving window

Downloaded as is and got the following error:

Traceback (most recent call last): File "main.py", line 44, in <module> dnc = DNC(Hp.batch_size, controller, task.vector_size, Hp.Mem, initializer=weight_initializer, initial_stddev=0.1) File "/home/maik/kaggle/src/controller_implementations/dnc/dnc.py", line 20, in __init__ self.controller_output_size = self.controller.out_vector_size AttributeError: 'int' object has no attribute 'out_vector_size'