thushv89 / exercises_thushv_dot_com Goto Github PK

how to inference - with your model

i train work embedding 300 dimension with 12000 word from https://nlp.stanford.edu/projects/nmt/data/iwslt15.en-vi/.
but not work.. help me tks you very much.

Started Training
.

InvalidArgumentError Traceback (most recent call last)
~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1360 try:
-> 1361 return fn(*args)
1362 except errors.OpError as e:

~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
1339 return tf_session.TF_Run(session, options, feed_dict, fetch_list,
-> 1340 target_list, status, run_metadata)
1341

~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py in exit(self, type_arg, value_arg, traceback_arg)
515 compat.as_text(c_api.TF_Message(self.status.status)),
--> 516 c_api.TF_GetCode(self.status.status))
517 # Delete the underlying status object from memory otherwise it stays alive

InvalidArgumentError: indices[5] = 12036 is not in [0, 12000)
[[Node: embedding_lookup_16 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@const"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Const, _arg_enc_train_inputs_16_0_154)]]

During handling of the above exception, another exception occurred:

InvalidArgumentError Traceback (most recent call last)
in ()
51 # ======================= OPTIMIZATION ==========================
52 if step < 10000:
---> 53 _,l,tr_pred = sess.run([adam_optimize,loss,train_prediction], feed_dict=feed_dict)
54 else:
55 _,l,tr_pred = sess.run([sgd_optimize,loss,train_prediction], feed_dict=feed_dict)

~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
903 try:
904 result = self._run(None, fetches, feed_dict, options_ptr,
--> 905 run_metadata_ptr)
906 if run_metadata:
907 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1135 if final_fetches or final_targets or (handle and feed_dict_tensor):
1136 results = self._do_run(handle, final_targets, final_fetches,
-> 1137 feed_dict_tensor, options, run_metadata)
1138 else:
1139 results = []

~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1353 if handle is None:
1354 return self._do_call(_run_fn, self._session, feeds, fetches, targets,
-> 1355 options, run_metadata)
1356 else:
1357 return self._do_call(_prun_fn, self._session, handle, feeds, fetches)

~/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1372 except KeyError:
1373 pass
-> 1374 raise type(e)(node_def, op, message)
1375
1376 def _extend_graph(self):

InvalidArgumentError: indices[5] = 12036 is not in [0, 12000)
[[Node: embedding_lookup_16 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@const"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Const, _arg_enc_train_inputs_16_0_154)]]

Caused by op 'embedding_lookup_16', defined at:
File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/root/tensorflow-dev/lib/python3.5/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 486, in start
self.io_loop.start()
File "/root/tensorflow-dev/lib/python3.5/site-packages/tornado/platform/asyncio.py", line 127, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 345, in run_forever
self._run_once()
File "/usr/lib/python3.5/asyncio/base_events.py", line 1312, in _run_once
handle._run()
File "/usr/lib/python3.5/asyncio/events.py", line 125, in _run
self._callback(*self._args)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tornado/platform/asyncio.py", line 117, in _handle_events
handler_func(fileobj, events)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tornado/stack_context.py", line 276, in null_wrapper
return fn(*args, **kwargs)
File "/root/tensorflow-dev/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 450, in _handle_events
self._handle_recv()
File "/root/tensorflow-dev/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 480, in _handle_recv
self._run_callback(callback, msg)
File "/root/tensorflow-dev/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 432, in _run_callback
callback(*args, **kwargs)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tornado/stack_context.py", line 276, in null_wrapper
return fn(*args, **kwargs)
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/root/tensorflow-dev/lib/python3.5/site-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/root/tensorflow-dev/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2662, in run_cell
raw_cell, store_history, silent, shell_futures)
File "/root/tensorflow-dev/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2785, in _run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/root/tensorflow-dev/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2903, in run_ast_nodes
if self.run_code(code, result):
File "/root/tensorflow-dev/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 23, in
encoder_emb_inp = [tf.nn.embedding_lookup(encoder_emb_layer, src) for src in enc_train_inputs]
File "", line 23, in
encoder_emb_inp = [tf.nn.embedding_lookup(encoder_emb_layer, src) for src in enc_train_inputs]
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/ops/embedding_ops.py", line 327, in embedding_lookup
transform_fn=None)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/ops/embedding_ops.py", line 151, in _embedding_lookup_and_transform
result = _clip(_gather(params[0], ids, name=name), ids, max_norm)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/ops/embedding_ops.py", line 55, in _gather
return array_ops.gather(params, ids, name=name)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 2667, in gather
params, indices, validate_indices=validate_indices, name=name)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1777, in gather
validate_indices=validate_indices, name=name)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3271, in create_op
op_def=op_def)
File "/root/tensorflow-dev/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1650, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): indices[5] = 12036 is not in [0, 12000)
[[Node: embedding_lookup_16 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@const"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Const, _arg_enc_train_inputs_16_0_154)]]

Hi,

Im trying to train my own data. I prepared source, target, vocab_source, target_source but i stuck on to create embeddings file. How can i do it?

The nmt_tutorial.ipynb notebook requires the vocab.50K.de.txt file. "FileNotFoundError" is thrown while running the notebook.

Hi @thushv89 ,

Thanks for your tutorial on neural machine translation. I am a newbie to this field and getting stuck in the data pre-processing part of the code.

def split_to_tokens(sent,is_source):
#sent = sent.replace('-',' ')
sent = sent.replace(',',' ,')
sent = sent.replace('.',' .')
sent = sent.replace('\n',' ')
sent_toks = sent.split(' ')

Can you explain me this part? Why have you replaced newline character with a space . Shouldn't that be denoting end of sentence () or in this case.
Or if replacing \n with a space , shouldn't split be done according to the full stop ' .' ??

Also, why should target sentences start and end from ?

Thanks a lot!

There are a couple of peculiar 'if' statements that I would like to be checked for accuracy (they don't appear to do anything):

                if sent_text[self._cursor[b]]!=src_dictionary['<s>']:
                    batch_data[b] = sent_text[self._cursor[b]]
                else:
                    batch_data[b] = sent_text[self._cursor[b]]

            if self._is_source:
                data, labels = self.next_batch(self._sent_ids, False)
            else:
                data, labels = self.next_batch(self._sent_ids, False)

Hello @thushv89 , thanks for the great materials. I am trying to adapt the seq2seq NMT code as an autoencoder for sentence embedding. Do you think it's a good way to use it for such purpose?

Thanks a lot!