Test model after saving with torch.save by loading glove weights about pytorch-sentiment-analysis HOT 3 CLOSED

Wait, once I load the model, I can see the embedding layer already fine tuned. This mapping of stoi and itos while take place from this layer right? Correct me if I'm wrong.

from pytorch-sentiment-analysis.

You don't need to re-load the word embedding parameters with model.embedding.weight.data.copy_(TEXT.vocab.vectors) as they will already be saved with the model when you do torch.save.

The only issue is you need to make sure the vocabulary of your new data is exactly the same as the vocabulary of your old data.

Say your original vocab stoi was {'the': 0, 'cat': 1, 'sat': 2} and your vocab stoi for the new data was {'the': 0, 'bird': 1, 'flew': 2}. Using your new data/vocab, if you index the sentence "the bird flew" you will get a tensor of [0, 1, 2]. However, when you pass this tensor through the embedding layer, you will actually get the embeddings for the sentence "the cat sat".

One way to get around this is by finishing up your training on the first set of data by turning the vocab and corresponding tensors into a dictionary, i.e. {'the': [0.2, 0.1, 0.3], 'cat': [0.3, 0.6, -0.2], ... and then saving it (as a .json file or similar). Then, when using your new data, you'll create your vocab without using pre-trained vectors, and then create a function that loads the saved vocab-tensor dictionary and iterates over all the keys. If you find a key in the loaded vocab that is also in your current vocab you can initialize that column of the embedding weight tensor (indicated by the stoi value) with your loaded vocab-tensor value.

Let me know if that isn't clear and make a quick gist.

from pytorch-sentiment-analysis.

Yes, I understood. I was loading the same train set into testing and saw the mismatch of stoi/itos. That makes perfect sense. Thanks a lot for making it clear.
I have another question about one-hot encoded targets. Ill generate a new issue.

from pytorch-sentiment-analysis.