lajanugen / zeshel Goto Github PK

Greetings!

When running the scripts/create_training_data.sh script, I get the following KeyErrors:

Traceback (most recent call last):
File "create_training_data.py", line 430, in
tf.app.run()
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "create_training_data.py", line 410, in main
rng, is_training=FLAGS.is_training)
File "create_training_data.py", line 210, in create_training_instances
vocab_words, rng, is_training=is_training)
File "create_training_data.py", line 295, in create_instances_from_document
context_document = all_documents[context_document_id]['text']
KeyError: 'A1DB81433C6D9C29'
Traceback (most recent call last):
File "create_training_data.py", line 430, in
tf.app.run()
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "create_training_data.py", line 410, in main
rng, is_training=FLAGS.is_training)
File "create_training_data.py", line 210, in create_training_instances
vocab_words, rng, is_training=is_training)
File "create_training_data.py", line 295, in create_instances_from_document
context_document = all_documents[context_document_id]['text']
KeyError: '1F92A2A6993DB564'
Traceback (most recent call last):
File "create_training_data.py", line 430, in
tf.app.run()
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "create_training_data.py", line 410, in main
rng, is_training=FLAGS.is_training)
File "create_training_data.py", line 210, in create_training_instances
vocab_words, rng, is_training=is_training)
File "create_training_data.py", line 295, in create_instances_from_document
context_document = all_documents[context_document_id]['text']
KeyError: '804387C91894D89D'

As a result, I'm not able to prepare the training data: only the validation and test data are able to be processed.

Any idea about why this is happening and how to correct it? Thanks in advance.

Hi @lajanugen,
Thanks for releasing the data and code! Great work.
I would like to ask about the preprocessing method used in the producing the dataset: what was the preprocessing method used to clean the raw text from Wikia? Any chance that the raw text can be released? Thanks.

When trying use run_classifier.sh I get the error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Key: segment_ids. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext]]
(1) Invalid argument: Key: segment_ids. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext]]
[[IteratorGetNext/_4055]]

I'm using reducing the sequence length and the batch size to attempt to fit into the 12GB of memory on the GPU I'm using with the following parameters:

python run_classifier.py
--do_train=true
--do_eval=false
--data_dir=$TFRecords
--vocab_file=$BERT_BASE_DIR/vocab.txt
--bert_config_file=$BERT_BASE_DIR/bert_config.json
--init_checkpoint=$INIT
--max_seq_length=128
--train_batch_size=4
--learning_rate=2e-5
--num_train_epochs=3.0
--num_cands=64
--save_checkpoints_steps=6000
--output_dir=$EXPTS_DIR/$EXP_NAME
--use_tpu=$USE_TPU \

Thanks for uploading codes for Zero-shot EL, I'm very interested in this work.
Currently I'm reading codes but can't find where mention position is indicated.
(In the original paper, written as

[Ms], [Me] are special tokens to tag the mention.

)

https://github.com/lajanugen/zeshel/blob/master/create_training_data.py#L351 indicates [CLS] and [SEP] for concatenating mention-contexts and candidate documents.
I can't find codes for inverting [Ms] and [Me].

If you'd know something about this, I'd appreciate it much.
Thanks.

Hello @lajanugen , thanks for your works. Could you please provide your pre-trained BERT models? I found that results will be much higher after using pre-trained models. Due to limited GPU resources, I could not pre train it. Thanks in advance！

Hi when I run the following command:

ZESHEL_DATA="MY DATA PATH" bash scripts/create_training_data.sh

I met the following error

 File "/scratch/yerong/local/anaconda3/envs/blink37/lib/python3.7/site-packages/tensorflow_core/python/lib/io/file_io.py", line 84, in _preread_check
compat.as_bytes(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.NotFoundError: uncased_L-12_H-768_A-12/vocab.txt; No such file or directory

Anyone knows how to fix this?

python3.7
bert-tensorflow        1.0.4
tensorflow             1.15.0

Dear Lajanugen,

I notice your paper as soon as the accepted paper list of ACL'19 is available. What a novel approach and great work!

I have two questions about the training procedure.
Q1: What do you mean by "the model is trained using a softmax loss"? Is it referring to the Cross Entropy Loss?
Q2: Do you concat the unseen classes with seen classes during training? (but no samples for unseen classes available).

Thanks and best wishes for you,
Ramon

The dataset includes the documents and titles, but not the corresponding web pages they come from. I'm wondering if you have those url available that can be shared.

Hi,

Thanks for your work.
Could you please provide your best model for use of your system without re-training?
Or, could you please provide the systems' outputs for analysis usage?

Thanks a lot.
Best