when i want to generate text i get a memory error
Traceback (most recent call last): File "rnn_tf.py", line 300, in <module> main() File "rnn_tf.py", line 221, in main data, vocab = load_data(args.input_file) File "rnn_tf.py", line 174, in load_data data = embed_to_vocab(data_, vocab) File "rnn_tf.py", line 152, in embed_to_vocab data = np.zeros((len(data_), len(vocab))) MemoryError
the text file is ~6000 KB in size but that shouldn't be a problem because i can train with this text.
i am running python in 64-bit
please help!

First thanks for this work. Works very well.
Whilst I'm testing it, it issues a warning:

WARNING:tensorflow:<tensorflow.python.ops.rnn_cell_impl.BasicLSTMCell object at 0x119c13748>: Using a concatenated state is slower and will soon be deprecated.  Use state_is_tuple=True.

tensorflow version 1.3.0
Python 3.6.2

Anyway to 'fix' it?

I know my last issue was my fault, but I don't think this one is. I got an argument error about a minute after starting the program. Here's the full error:

Traceback (most recent call last):
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_call
    return fn(*args)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1329, in _run_fn
    status, run_metadata)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,67] rhs shape= [256,39]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@char_rnn_network/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rnn_network/Variable, save/RestoreV2)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "rnn_tf.py", line 309, in <module>
    main()
  File "rnn_tf.py", line 261, in main
    check_restore_parameters(sess, saver)
  File "rnn_tf.py", line 191, in check_restore_parameters
    saver.restore(sess, ckpt.model_checkpoint_path)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1686, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
    run_metadata_ptr)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1344, in _do_run
    options, run_metadata)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,67] rhs shape= [256,39]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@char_rnn_network/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rnn_network/Variable, save/RestoreV2)]]

Caused by op 'save/Assign', defined at:
  File "rnn_tf.py", line 309, in <module>
    main()
  File "rnn_tf.py", line 257, in main
    saver = tf.train.Saver(tf.global_variables())
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1239, in __init__
    self.build()
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1248, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1284, in _build
    build_save=build_save, build_restore=build_restore)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 765, in _build_internal
    restore_sequentially, reshape)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 440, in _AddRestoreOps
    assign_ops.append(saveable.restore(tensors, shapes))
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 160, in restore
    self.op.get_shape().is_fully_defined())
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\ops\state_ops.py", line 276, in assign
    validate_shape=validate_shape)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 62, in assign
    use_locking=use_locking, name=name)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3160, in create_op
    op_def=op_def)
  File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1625, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [256,67] rhs shape= [256,39]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@char_rnn_network/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rnn_network/Variable, save/RestoreV2)]]

I tried to generate German text. Unfortunately the results are only extracts from the original text. But no generated new remixes.

For testing i used Wagner's Lohengrin (old German) and German Hip Hop lyrics (of one group). In order to verify i successfully tried Vernor Vinges Rainbows End in English language. #

I notice that the longer(the more batches)that I've trained the more perfect that generated sentences will be,but I find that some generated sentences can be totally the same as some sentences in my train corpus,is it possible?I just wonder whether it just generate the sentences like that or 'copy' like that.

Nice work! It is very clear and easy to understand!

I am wondering how could I change it to a word-level Generator?

I didn't see so quickly any other way to contact you, but I wanted to give credit. I re-used your model with some slight adjustments to generate icons and hieroglyphs instead of text:

http://blog.douweosinga.com/2017/01/learning-to-draw-generating-icons-and.html

I wanted to train with a fairly large training data file (~60MB). However, I get MemoryErrors. I found out your call to np.zeros() was trying to create an array of 197,159,216,917 elements. Seeing that I don't have 200GB of RAM/Swap, I could see why this would cause a problem. Is there any way to reduce this massive use of memory?

As you can see after i've terminated the program it doesn't output anything but multiple errors. Same thing happens when i run
$ python rnn_tf.py saved/model.ckpt "The "
although i doesn't happen when i run this code:
$ python rnn_tf.py
another thing that doesn't work is the relative position of the file. So i always have to give the absolute position of data/shakespeare.txt. All of the program runs in terminal Mac version 10.12. İ use python 3.6.2.
İf you need any other information please just ask.
Thank you in advance!

Hi I am trying to install and I get
Traceback (most recent call last):
File "rnn_tf.py", line 5, in
import argparse
File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 86, in
import copy as _copy
File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy.py", line 52, in
import weakref
File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/weakref.py", line 14, in
from _weakref import (
ImportError: cannot import name _remove_dead_weakref

Hi. Can I use it to generate text including russian words/language? Will it work correctly?

And can u explain me a lil' bit more about how to start generate text?

Cuz' when i choose new textfile to train and started with a command 'python rnn_tf.py' - I have been taking some OpKernels and warnigs + after that waiting for a long time (something like 2-3 hours). There was 800 batches and they didn't want to stop. I stoppped it by myself, cuz' losses was higher, then previos 700 batches.

After that i started with a command ' python rnn_ft.py saved/model.ckpt "The " ' and programm returned me some errors.

Did i did everything right? Mb i should waight till train will be complete? Is it normal, that losses was growing up after 700 batches?

Sorry for my stupid questions and thanks :)

Hi spiglerg!

First off, I want to say that your code has helped me become much more familiar with the RNN framework inside of Tensorflow. Thank you!

I noticed that on line 47 of rnn_tf.py you create a two-element list of LSTM cells by saying:
[self.lstm_cell]*self.num_layers
Which creates a list of two of the exact same TF objects. Since each TF object has its own set of trainable parameters, won't duplicating the same TF object cause the network to have two layers of LSTM cells that have the exact same parameters?

Hi. It's me again :)

K, i used command $ python rnn_tf.py and some hours ago it was finished. And what should i do now?
Should i use command $ python rnn_tf.py saved/model.ckpt "The " to generate my text?
Will it generate a new file at "saved" directory ? How can i find text file?
Or mb i need to use another prefix? I didn't understand it yet :( Can u explain me a little, please?

Thx a lot and sorry for giving u some work :)

Hi,

First, thanks for your work :)
Second, I tried to use your code without any changes for test purposes,, but result isn't good.

Output of python rnn_tf.py saved\model.ckpt "The "

It's a normal result for default configuration or not?

Btw, one time I tried to clear saved folder, and after train file model.ckpt didn't create, only checkpoint/index/meta/data files. It's right behavior or I missed something?

My env:
Windows 10 x64
Python 3.5.2
Tensorflow GPU 1.1.0 (works with GeForce GTX 950M)

Hi:

Your script is generating very good training errors below 0.4. after 60k batches Is there a snippet of code that can be added to include a temperature option common to other lstm packages to improve output?

Thanks