Hello, I am unable to use the ELMo implementation even though I follow the arguments provided at README. I use Python 3.6.2 (Anaconda) and install AllenNLP on virtual environment
GPUID=-1
SEED=19
SHOULD_TRAIN=1
WORD_EMBS_FILE="../glove/glove.6B/glove.6B.50d.txt"
d_word=50
d_hid=512
glove=0
ELMO=1
deep_elmo=0
elmo_no_glove=1
COVE=0
PAIR_ENC="simple"
(allennlp) ➜ bash run_stuff.sh
12/01 04:00:19 PM: Namespace(batch_size=64, bpp_base=10, bpp_method='percent_tr', classifier='mlp', classifier_dropout=0.0, classifier_hid_dim=512, cove=0, cuda=-1, d_hid=512, d_word=50, deep_elmo=0, dropout=0.2, dropout_embs=0.2, elmo=1, elmo_no_glove=1, eval_tasks='none', exp_dir='EXP_DIR', glove=0, load_epoch=-1, load_model=0, load_preproc=1, load_tasks=1, log_file='log.log', lr=0.1, lr_decay_factor=0.5, max_grad_norm=5.0, max_seq_len=40, max_vals=100, max_word_v_size=30000, min_lr=1e-05, n_epochs=10, n_layers_enc=1, n_layers_highway=0, no_tqdm=0, optimizer='sgd', pair_enc='simple', patience=5, preproc_file='preproc.pkl', random_seed=19, run_dir='RUN_DIR', scaling_method='none', scheduler_threshold=0.0, shared_optimizer=1, should_train=1, task_ordering='random', task_patience=0, train_tasks='cola', train_words=0, trainer_type='sampling', val_interval=10, weight_decay=0.0, weighting_method='uniform', word_embs_file='../glove/glove.6B/glove.6B.50d.txt')
12/01 04:00:19 PM: Using random seed 19
12/01 04:00:19 PM: Loading tasks...
12/01 04:00:19 PM: Loaded existing task cola
12/01 04:00:19 PM: Loaded existing task sst
12/01 04:00:19 PM: Loaded existing task mrpc
12/01 04:00:19 PM: Finished loading tasks: cola sst mrpc.
12/01 04:00:22 PM: Loading token dictionary from EXP_DIR/vocab.
12/01 04:00:22 PM: Finished building vocab. Using 30002 words
12/01 04:00:22 PM: Loaded data from EXP_DIR/preproc.pkl
12/01 04:00:22 PM: Training on cola, sst, mrpc
12/01 04:00:22 PM: Evaluating on
12/01 04:00:22 PM: Finished loading tasks in 3.215s
12/01 04:00:22 PM: Building model...
12/01 04:00:22 PM: Learning embeddings from scratch!
12/01 04:00:22 PM: Using ELMo embeddings!
12/01 04:00:22 PM: NOT using GLoVe embeddings!
12/01 04:00:22 PM: Initializing ELMo
12/01 04:00:43 PM: instantiating registered subclass lstm of <class 'allennlp.modules.seq2seq_encoders.seq2seq_encoder.Seq2SeqEncoder'>
12/01 04:00:43 PM: batch_first = True
12/01 04:00:43 PM: stateful = False
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: input_size = 1024
12/01 04:00:43 PM: hidden_size = 512
12/01 04:00:43 PM: num_layers = 1
12/01 04:00:43 PM: bidirectional = True
12/01 04:00:43 PM: batch_first = True
12/01 04:00:43 PM: Initializing parameters
12/01 04:00:43 PM: Done initializing parameters; the following parameters are using their default initialization from their code
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._char_embedding_weights
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.0.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.0.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.1.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.1.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._projection.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_0.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_0.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_1.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_1.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_2.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_2.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_3.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_3.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_4.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_4.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_5.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_5.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_6.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_6.weight
12/01 04:00:43 PM: _elmo.scalar_mix_0.gamma
12/01 04:00:43 PM: _elmo.scalar_mix_0.scalar_parameters.0
12/01 04:00:43 PM: _elmo.scalar_mix_0.scalar_parameters.1
12/01 04:00:43 PM: _elmo.scalar_mix_0.scalar_parameters.2
12/01 04:00:43 PM: _phrase_layer._module.bias_hh_l0
12/01 04:00:43 PM: _phrase_layer._module.bias_hh_l0_reverse
12/01 04:00:43 PM: _phrase_layer._module.bias_ih_l0
12/01 04:00:43 PM: _phrase_layer._module.bias_ih_l0_reverse
12/01 04:00:43 PM: _phrase_layer._module.weight_hh_l0
12/01 04:00:43 PM: _phrase_layer._module.weight_hh_l0_reverse
12/01 04:00:43 PM: _phrase_layer._module.weight_ih_l0
12/01 04:00:43 PM: _phrase_layer._module.weight_ih_l0_reverse
12/01 04:00:43 PM: Initializing parameters
12/01 04:00:43 PM: Done initializing parameters; the following parameters are using their default initialization from their code
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_0.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.backward_layer_1.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_0.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.input_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.state_linearity.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.state_linearity.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._elmo_lstm.forward_layer_1.state_projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._char_embedding_weights
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.0.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.0.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.1.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._highways._layers.1.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._projection.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder._projection.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_0.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_0.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_1.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_1.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_2.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_2.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_3.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_3.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_4.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_4.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_5.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_5.weight
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_6.bias
12/01 04:00:43 PM: _elmo._elmo_lstm._token_embedder.char_conv_6.weight
12/01 04:00:43 PM: _elmo.scalar_mix_0.gamma
12/01 04:00:43 PM: _elmo.scalar_mix_0.scalar_parameters.0
12/01 04:00:43 PM: _elmo.scalar_mix_0.scalar_parameters.1
12/01 04:00:43 PM: _elmo.scalar_mix_0.scalar_parameters.2
12/01 04:00:43 PM: _phrase_layer._module.bias_hh_l0
12/01 04:00:43 PM: _phrase_layer._module.bias_hh_l0_reverse
12/01 04:00:43 PM: _phrase_layer._module.bias_ih_l0
12/01 04:00:43 PM: _phrase_layer._module.bias_ih_l0_reverse
12/01 04:00:43 PM: _phrase_layer._module.weight_hh_l0
12/01 04:00:43 PM: _phrase_layer._module.weight_hh_l0_reverse
12/01 04:00:43 PM: _phrase_layer._module.weight_ih_l0
12/01 04:00:43 PM: _phrase_layer._module.weight_ih_l0_reverse
12/01 04:00:43 PM: Finished building model in 20.876s
12/01 04:00:43 PM: patience = 5
12/01 04:00:43 PM: num_epochs = 10
12/01 04:00:43 PM: max_vals = 50
12/01 04:00:43 PM: cuda_device = -1
12/01 04:00:43 PM: grad_norm = 5.0
12/01 04:00:43 PM: grad_clipping = None
12/01 04:00:43 PM: lr_decay = 0.99
12/01 04:00:43 PM: min_lr = 1e-05
12/01 04:00:43 PM: no_tqdm = 0
12/01 04:00:43 PM: Sampling tasks uniformly
12/01 04:00:43 PM: type = sgd
12/01 04:00:43 PM: parameter_groups = None
12/01 04:00:43 PM: Number of trainable parameters: 9449994
12/01 04:00:43 PM: instantiating registered subclass sgd of <class 'allennlp.training.optimizers.Optimizer'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: lr = 0.1
12/01 04:00:43 PM: weight_decay = 1e-05
12/01 04:00:43 PM: type = reduce_on_plateau
12/01 04:00:43 PM: instantiating registered subclass reduce_on_plateau of <class 'allennlp.training.learning_rate_schedulers.LearningRateScheduler'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: mode = max
12/01 04:00:43 PM: factor = 0.5
12/01 04:00:43 PM: patience = 0
12/01 04:00:43 PM: threshold = 0.0
12/01 04:00:43 PM: threshold_mode = abs
12/01 04:00:43 PM: verbose = True
12/01 04:00:43 PM: type = sgd
12/01 04:00:43 PM: parameter_groups = None
12/01 04:00:43 PM: Number of trainable parameters: 9449994
12/01 04:00:43 PM: instantiating registered subclass sgd of <class 'allennlp.training.optimizers.Optimizer'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: lr = 0.1
12/01 04:00:43 PM: weight_decay = 1e-05
12/01 04:00:43 PM: type = reduce_on_plateau
12/01 04:00:43 PM: instantiating registered subclass reduce_on_plateau of <class 'allennlp.training.learning_rate_schedulers.LearningRateScheduler'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: mode = max
12/01 04:00:43 PM: factor = 0.5
12/01 04:00:43 PM: patience = 0
12/01 04:00:43 PM: threshold = 0.0
12/01 04:00:43 PM: threshold_mode = abs
12/01 04:00:43 PM: verbose = True
12/01 04:00:43 PM: type = sgd
12/01 04:00:43 PM: parameter_groups = None
12/01 04:00:43 PM: Number of trainable parameters: 9449994
12/01 04:00:43 PM: instantiating registered subclass sgd of <class 'allennlp.training.optimizers.Optimizer'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: lr = 0.1
12/01 04:00:43 PM: weight_decay = 1e-05
12/01 04:00:43 PM: type = reduce_on_plateau
12/01 04:00:43 PM: instantiating registered subclass reduce_on_plateau of <class 'allennlp.training.learning_rate_schedulers.LearningRateScheduler'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: mode = max
12/01 04:00:43 PM: factor = 0.5
12/01 04:00:43 PM: patience = 0
12/01 04:00:43 PM: threshold = 0.0
12/01 04:00:43 PM: threshold_mode = abs
12/01 04:00:43 PM: verbose = True
12/01 04:00:43 PM: type = sgd
12/01 04:00:43 PM: parameter_groups = None
12/01 04:00:43 PM: Number of trainable parameters: 9449994
12/01 04:00:43 PM: instantiating registered subclass sgd of <class 'allennlp.training.optimizers.Optimizer'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: lr = 0.1
12/01 04:00:43 PM: weight_decay = 1e-05
12/01 04:00:43 PM: type = reduce_on_plateau
12/01 04:00:43 PM: instantiating registered subclass reduce_on_plateau of <class 'allennlp.training.learning_rate_schedulers.LearningRateScheduler'>
12/01 04:00:43 PM: Converting Params object to dict; logging of default values will not occur when dictionary parameters are used subsequently.
12/01 04:00:43 PM: CURRENTLY DEFINED PARAMETERS:
12/01 04:00:43 PM: mode = max
12/01 04:00:43 PM: factor = 0.5
12/01 04:00:43 PM: patience = 0
12/01 04:00:43 PM: threshold = 0.0
12/01 04:00:43 PM: threshold_mode = abs
12/01 04:00:43 PM: verbose = True
12/01 04:00:43 PM: Beginning training.
Traceback (most recent call last):
File "main.py", line 280, in <module>
sys.exit(main(sys.argv[1:]))
File "main.py", line 177, in main
args.load_model)
File "/Users/apple/Desktop/q1_course/CS273/ml_final/CS273A/src/trainer.py", line 776, in train
output_dict = self._forward(batch, task=task, for_training=True)
File "/Users/apple/Desktop/q1_course/CS273/ml_final/CS273A/src/trainer.py", line 1003, in _forward
return self._model.forward(task, **tensor_batch)
File "/Users/apple/Desktop/q1_course/CS273/ml_final/CS273A/src/models.py", line 216, in forward
pair_emb = self.pair_encoder(input1, input2)
File "/Users/apple/anaconda3/envs/allennlp/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/Users/apple/Desktop/q1_course/CS273/ml_final/CS273A/src/models.py", line 289, in forward
s1_elmo_embs = self._elmo(s1['elmo'])
KeyError: 'elmo'
If you suspect this is an IPython bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at [email protected]
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True