matthewinkawhich / pytorch-chatbot Goto Github PK

Sorry, I think I misunderstood the tutorial, thus I close the issue.

I'm following the chatbot tutorial at this URL.

https://pytorch.org/tutorials/beginner/chatbot_tutorial.html

I've written two Modules. One is a decoder and one is an encoder. They run. My problem is that when I look at my output, quite often I just see the word 'I' alone. In other words the chatbot thinks that it is sufficient to answer every question with the word 'I' alone. I donwloaded the code from the tutorial and that code does not show this behavior.

I've tried a tensorflow project, and though it was a long time ago I believe I remember having this same problem. I am using the movie database as my corpus. I have to think someone has seen this problem before.

My setup currently is that the encoder processes a whole batch at a time, while the decoder goes through the batch line by line and the lines word by word. Is this my problem?

my code changes often and is very messy, but the url to the seq2seq model on github is here:

https://github.com/radiodee1/awesome-chatbot/blob/master/model/seq_2_seq.py

any help would be appreciated.

Hello Matthew !

I am studying your code right now and learn that you implemented Luong et al.'s global attention, which uses current target hidden state and all encoder hidden state to calculate the score function. However, I noticed that in your code it was calculated by the current target output. So is this a variance from the paper or I am understanding it wrong?

# Calculate attention weights from the current GRU output
 attn_weights = self.attn(rnn_output, encoder_outputs)

Much appreciated your work. Thank you.

Hi Matthew,

First, thanks a lot for the Chatbot tutorial!

In the PyTorch discussion board, some raised a question with respect to a specific line of code:

# Set initial decoder hidden state to the encoder's final hidden state
decoder_hidden = encoder_hidden[:decoder.n_layers]

I would agree that this is not an intuitive way to do it; again, the post discusses all the details. Could you clarify if we are missing anything? Otherwise, I would argue that this line of code is not appropriate.

Thanks and best,

Chris