What does token_num parameter in get_model function mean? about keras-bert HOT 7 CLOSED

The size of your dictionary / The input dimension of the embedding

from keras-bert.

I am getting memory errors (used several GPUs with at least 12 gb ram) even for a small size (7k) of dictionary. Considering the real dictionary must consist 100k tokens, there is something wrong. I opened an issue at Keras repo, I will be really happy if you can take a look since it is based on your code.
link: keras-team/keras#12755

from keras-bert.

I think you misunderstood steps_per_epoch. If you are using a generator, then the batch size is controlled only by the generator itself. Setting steps_per_epoch means there are batch_size x steps_per_epochsamples in one epoch.

from keras-bert.

But don't we set number of sequences/samples? Doesn't this allow us control batch size by changing steps_per_epoch ?

from keras-bert.

There is no way to modify your generator "intelligently" since you may need operators like padding.

from keras-bert.

Okay. What do you suggest about deciding the value of steps_per_epoch ?

from keras-bert.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from keras-bert.