sordonia / hed-dlg Goto Github PK

The method "plain_step(self, xd_t, m_t, hs_t, hd_tm1)" in dialog.encdec.py uses the variable Wd_s_q, but this variable Wd_s_q is only defined for the gated step. I am not sure how to solve the problem, but until we solve it we cannot run the simple (un-gated) RNN. @sordonia

The decoder does not take only the hidden state of the dialog RNN, s_{t},
but also the last hidden state of the encoder RNN, h_{t}. Possibly, the decoder
should take the s_{t-1} and not s_{t}.

Refactor stochastic sampling and beam search to be independent of the model structure. This will prevent many future bugs and reduce duplicated code.

These two procedures should call a single function from the model to retrieve the complete model state (including encoders, context and decoder), a single function to sample (stochastically) or take argmax (beam search) and a single function to add the result back into the model state. @sordonia

Implement Variational Stochastic training.

When I set maxout_out = True for prototype_test Theano crashes because of an inconsistency between weight sizes. The prototype_moviedic works fine with maxout_out though...

The bug happens in the latest version of the code, but I think the exact same thing will happen in the older versions of the code. This is the error message I get:

Traceback (most recent call last):
File "train.py", line 550, in
main(args)
File "train.py", line 196, in main
samples, costs = random_sampler.sample([[]], n_samples=1, n_turns=3)
File "/home/julian/UdeM/Projects/Dialog/Github/hed-dlg/search.py", line 47, in sample_apply
samples, costs = sample_logic(sampler, joined_context, **kwargs)
File "/home/julian/UdeM/Projects/Dialog/Github/hed-dlg/search.py", line 168, in sample
next_probs, new_hd = self.next_probs_predictor(prev_hs, prev_words, prev_hd)
File "/home/julian/Theano/Theano/theano/compile/function_module.py", line 599, in call
outputs = self.fn()
File "/home/julian/Theano/Theano/theano/gof/link.py", line 526, in streamline_default_f
raise_with_op(node, thunk)
File "/home/julian/Theano/Theano/theano/gof/link.py", line 522, in streamline_default_f
thunk()
File "/home/julian/Theano/Theano/theano/gof/op.py", line 759, in rval
r = p(n, [x[0] for x in i], o)
File "/home/julian/Theano/Theano/theano/tensor/blas.py", line 1653, in perform
z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: ('shapes (1,25) and (10,17) not aligned: 25 (dim 1) != 10 (dim 0)', (1, 25), (10, 17))
Apply node that caused the error: Dot22(max, Wd_emb.T)
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(1, 25), (10, 17)]
Inputs strides: [(100, 4), (4, 40)]
Inputs values: ['not shown', 'not shown']

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

Shuffling in SS_dataset should be initialized with the same seed
used in the model in order to make the experiments reproducible.

• Word Perplexity
• Word Error Rate
• MRR

• BLEU
• Trigram Jaccard