atulkumarin / declare Goto Github PK

'exfake'?
https://github.com/huggingface/transformers

We should enforce this by default.

1. For model parameters
2. Anything else if we want to run everything as a script

hello, I want to reproduce the result in the paper; as the picture show, train_loss and train_accuracy looks good, but val_acc mostly do not change.
any advisice?

Expected Behavior

I am trying to run DeClarE using the DeClareMain.ipynb notebook in your repository.

Current Behavior

When it gets to the cell containing snopes = DeClareDataset(SNOPES_LOC, glove_data_file) I get the following error:

AttributeError Traceback (most recent call last)
in
----> 1 snopes = DeClareDataset(SNOPES_LOC, glove_data_file)

~/code/nlp_credibility/DeClare/data/datasets.py in init(self, news_dataset_path, glove_path)
35 self.vocab_vectors_path = os.path.join(os.path.dirname(news_dataset_path), 'vocab_vectors.npy')
36
---> 37 self._build_vocabulary()
38 self._build_source_vocabularies()
39

~/code/nlp_credibility/DeClare/data/datasets.py in _build_vocabulary(self)
76 self.vocab[word] = token_count
77 token_count += 1
---> 78 embeddings.append(self._vec(word))
79 else:
80 # Treat it as an unknown token

~/code/nlp_credibility/DeClare/data/datasets.py in _vec(self, w)
126
127 def _vec(self, w):
--> 128 return self.glove_df.loc[w].as_matrix()
129
130 def len(self):

~/miniconda/lib/python3.7/site-packages/pandas/core/generic.py in getattr(self, name)
5272 if self._info_axis._can_hold_identifiers_and_holds_name(name):
5273 return self[name]
-> 5274 return object.getattribute(self, name)
5275
5276 def setattr(self, name: str, value) -> None:

AttributeError: 'Series' object has no attribute 'as_matrix'

Possible Solution

I believe glove_df.loc is returning a _LocIndexer object which does not have an as_matrix() method. Could it be missing the method, or is self.glove_df.loc[w] supposed to return something different?

Steps to Reproduce

1. Running in order, the 4th cell down produces this error.

Although there will be no runtime issue due to the token in vocab, the model may not generalize well during inference. We can replace it with a universal vocab file (per-processced and fixed of appropriate size - some subset of GloVe vocabulary, taking the full Glove vocab will be memory intensive, so we can choose top frequent words on news sites).