pavelostyakov / toxic Goto Github PK

This is really a nice work.
And when I run this code. You know, embedding_utils.py this file, need to create very huge file or array. I have only 8 Gb RAM. So How much memory do I need? And any other suggestions?

def read_embedding_list(file_path):
    embedding_word_dict = {}
    embedding_list = []
    with open(file_path) as f:
        for row in tqdm.tqdm(f.read().split("\n")[1:-1]):
            data = row.split(" ")
            word = data[0]
            embedding = np.array([float(num) for num in data[1:-1]])
            embedding_list.append(embedding)
            embedding_word_dict[word] = len(embedding_word_dict)

    embedding_list = np.array(embedding_list)
    return embedding_list, embedding_word_dict

Can anyone tell me why the last element was cut?
like this one: data[1:-1]

Epoch 11 loss 0.04497938698477566 best_loss 0.0413083174741875
Predicting results...
Traceback (most recent call last):
File "fit_predict.py", line 124, in
main()
File "fit_predict.py", line 110, in main
test_predicts = np.multiply(*test_predicts_list)
ValueError: invalid number of arguments.

Did I miss something?

It will be great if you can provide the pre-trained model as well.

Hi,

I'm interested in using your extend_dataset.py script, but keep getting the same error:
urllib.error.HTTPError: HTTP Error 429: Too Many Requests
I'm trying the script on a dummy *.csv containing 100 English sentences, with all settings in their default. Any thoughts on how to circumvent this problem?

Thanks a lot!