Transformer decoder is giving the same out for every example about texar HOT 8 CLOSED

By my experience, the transformer decoder works well on other tasks. I personally haven't tried BERT as an encoder for generation.

What summarization data are you using?

Have you tried replacing transformer decoder with an RNN decoder for a quick comparison?

from texar.

I am using CNN data for building the model. I have not tried RNN decoder will try to do that .
But can you please check the code once in the given link if something is wrong.

from texar.

I was able to debug the the problem. Bert weights got disturbed and it was giving similar embeddings irrespective of example I am passing so the context of the sentence at the encoder is not correctly captured. Is there a way where I can set trainable as false for Bert based encoder directly?

from texar.

You can use tf.stop_gradient to disable gradient backpropagation to BERT, or specify the variables argument when calling tf.contrib.layers.optimize_loss to exclude BERT variables

from texar.

I am still facing the same issue.
Can you please elaborate on how you solved it?

from texar.

Is this the correct way?
hparams=tf.stop_gradient(hparams)

from texar.

This seems to be working:
encoder_output = tf.stop_gradient(encoder_output)
Is this the right way ?

from texar.

allvars = tf get_trainable_variables()
nonBert =[v for v in allvars if 'bert' not in v]

train_op = tx.core.get_train_op(
mle_loss,
learning_rate=learning_rate,
variables=non Bert,
global_step=global_step,
hparams=opt)

from texar.