currie32 / text-summarization-with-amazon-reviews Goto Github PK
View Code? Open in Web Editor NEWA seq2seq model that can generate summaries from fine food reviews on Amazon.
A seq2seq model that can generate summaries from fine food reviews on Amazon.
@Currie32 Can you please help - this is producing the same summary for each review. in my case repeating "not" many times.
What can be the root cause
In your code it seems like you are trying to create a multilayer encoder, however what is actually happening is that multiple single layer decoders are being created. As you can see in this graph image created by tensorflow http://imgur.com/a/jqAn5 the rnn inputs are feed into each bidirectional rnn rather than one feeding into the other. I created this image by running your encoding_layer
function and passing in some placeholders and then using the tf.summary.FileWriter
to draw the graph.
To create a multilayer encoder you would want something like this
def encoding_layer(rnn_size, sequence_length, num_layers, rnn_inputs, keep_prob):
'''Create the encoding layer'''
layer_input = rnn_inputs
for layer in range(num_layers):
with tf.variable_scope('encoder_{}'.format(layer)):
cell_fw = tf.contrib.rnn.LSTMCell(rnn_size,
initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
cell_fw = tf.contrib.rnn.DropoutWrapper(cell_fw,
input_keep_prob = keep_prob)
cell_bw = tf.contrib.rnn.LSTMCell(rnn_size,
initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
cell_bw = tf.contrib.rnn.DropoutWrapper(cell_bw,
input_keep_prob = keep_prob)
enc_output, enc_state = tf.nn.bidirectional_dynamic_rnn(cell_fw,
cell_bw,
layer_input,
sequence_length,
dtype=tf.float32)
layer_input = tf.concat(enc_output, 2)
# Join outputs since we are using a bidirectional RNN
enc_output = tf.concat(enc_output,2)
return enc_output, enc_state
You can see in this graph image http://imgur.com/a/bJANa That this creates the multi layer structure I assume you are going for. (The caveat is that in this function your rnn size must be half of the embedding size so if you want them to be different just move the first birnn out of the loop and have the layers after the first have the same size)
Thank you very much your tutorial. it really helped me a lot.
I am wondering if you can help me with this. In you set up, it seems that you are making every encoding layer as bidirectional. But i think it is also very common that we only make 1 bidirectional layer and making other layers as regular rnn layers.(like google's translation model).
I am not sure how to connect a bidirectional layer with a regular rnn layer?
Hi,
I'm trying to restore the session for the last checkpoint saved. I do this uncommenting this two lines of the code:
loader = tf.train.import_meta_graph("./" + checkpoint + '.meta')
loader.restore(sess, checkpoint)
But I'm getting this error:
ValueError: cannot add op with name encoder_0/bidirectional_rnn/fw/lstm_cell/kernel/Adam as that name is already used
Am I doing something wrong or is this an issue?
Hi, I have trained with Fine Food Reviews data with 100 epochs. After that I have changed dataset. downloaded dataset from amazon reviews dataset. There I have taken Home and Kitchen dataset in that dataset I have taken Review Text and Summary. And trained with 100 epochs.
After completion of training I have tested the model. getting same summary for distinct reviews. I have trained so many times getting the same summary while I am testing
The model made some good predictions when I tested it immediately after training was finished. But when I loaded the model after opening the notebook next day, it makes some random meaningless predictions.
Any reason for this?
Anyone else having difficulty opening the numberbatch-en-17.02.txt.
Please provide guidance on how you were able to extract it.
AttributeError: module 'tensorflow' has no attribute 'placeholder'
Hi ,i trained the model but when i tested with different reviews its giving same summary for all whatever we got when i executed first review ,i didnt understand whats the problem, can i know what is the reason for that.
Are they random vectors?
hi, when I use a similar code to solve a seq2seq problem, during the training, the loss decrease very quickly. but during the testing, all the output sequence are same. Can you example what sample_id means?
Here is the encoding_layer
for ease:
def encoding_layer(rnn_size, sequence_length, num_layers, rnn_inputs, keep_prob):
'''Create the encoding layer'''
for layer in range(num_layers):
with tf.variable_scope('encoder_{}'.format(layer)):
cell_fw = tf.contrib.rnn.LSTMCell(rnn_size,
initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
cell_fw = tf.contrib.rnn.DropoutWrapper(cell_fw,
input_keep_prob = keep_prob)
cell_bw = tf.contrib.rnn.LSTMCell(rnn_size,
initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
cell_bw = tf.contrib.rnn.DropoutWrapper(cell_bw,
input_keep_prob = keep_prob)
enc_output, enc_state = tf.nn.bidirectional_dynamic_rnn(cell_fw,
cell_bw,
rnn_inputs,
sequence_length,
dtype=tf.float32)
# Join outputs since we are using a bidirectional RNN
enc_output = tf.concat(enc_output,2)
return enc_output, enc_state
I'm new to tensorflow
, so here's my understanding:
Based on num_layers
, it'll create that many tf.nn.bidirectional_dynamic_rnn
s. So shouldn't the line enc_output = tf.concat(enc_output,2)
be inside the for loop ? Will the return statement returns all the enc_output
. If so how ?
And how to find accuracy, can you suggest briefly ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.