Giter VIP home page Giter VIP logo

kgreasoning's Introduction

KGReasoning

This repo contains several algorithms for multi-hop reasoning on knowledge graphs, including the official PyTorch implementation of Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs and a PyTorch implementation of Complex Query Answering with Neural Link Predictors.

Models

KG Data

The KG data (FB15k, FB15k-237, NELL995) mentioned in the BetaE paper and the Query2box paper can be downloaded here. Note the two use the same training queries, but the difference is that the valid/test queries in BetaE paper have a maximum number of answers, making it more realistic.

Each folder in the data represents a KG, including the following files.

  • train.txt/valid.txt/test.txt: KG edges
  • id2rel/rel2id/ent2id/id2ent.pkl: KG entity relation dicts
  • train-queries/valid-queries/test-queries.pkl: defaultdict(set), each key represents a query structure, and the value represents the instantiated queries
  • train-answers.pkl: defaultdict(set), each key represents a query, and the value represents the answers obtained in the training graph (edges in train.txt)
  • valid-easy-answers/test-easy-answers.pkl: defaultdict(set), each key represents a query, and the value represents the answers obtained in the training graph (edges in train.txt) / valid graph (edges in train.txt+valid.txt)
  • valid-hard-answers/test-hard-answers.pkl: defaultdict(set), each key represents a query, and the value represents the additional answers obtained in the validation graph (edges in train.txt+valid.txt) / test graph (edges in train.txt+valid.txt+test.txt)

We represent the query structures using a tuple in case we run out of names :), (credits to @michiyasunaga). For example, 1p queries: (e, (r,)) and 2i queries: ((e, (r,)),(e, (r,))). Check the code for more details.

Examples

Please refer to the examples.sh for the scripts of all 3 models on all 3 datasets.

Citations

If you use this repo, please cite the following paper.

@inproceedings{
 ren2020beta,
 title={Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs},
 author={Hongyu Ren and Jure Leskovec},
 booktitle={Neural Information Processing Systems},
 year={2020}
}

kgreasoning's People

Contributors

pminervini avatar hyren avatar roks avatar

Stargazers

Fred Xu avatar avatar Yin Hang avatar avatar avatar avatar yyyyynz avatar hhh avatar Peter Clarke avatar Haotong DU 杜昊桐 avatar Zifeng Ding avatar WangZhen avatar Eunhwan Park avatar yaoyaoyao avatar Max Berrendorf avatar DimitrisAlivas avatar Weiyu Liu avatar Jia-Ying Lin avatar KAIXIANG LIN avatar Tongzhou Mu avatar

Watchers

James Cloos avatar

Forkers

kiddozhu judepark96 blidge zahidabasher

kgreasoning's Issues

Error while running CQD model.

Could you offer a command to train and test CQD model?
I have tried to train CQD model with this command:

main.py --cuda --do_train --do_valid --do_test --data_path=data/FB15k-237-betae --print_on_screen --use-qa-iterator -n=128 -b=512 -d=800 -g=24 -lr=0.0001 --max_steps=450001 --cpu_num=1 --geo=cqd --valid_steps=15000 --tasks=1p

But it reported that RuntimeError: Adam does not support sparse gradients, please consider SparseAdam instead

Is this as expected? If it is, can I only replace Adam with SparseAdam?

Thanks~

Data explaination

Dear @pminervini!

Thanks for this awesome implementation of CQD. I appreciate the information you provided about the dataset. However, I would like to inquire further about the representation of relationships and entities in the dataset.

I noticed that some of the entity IDs have a "-2" appended to them in the following sequence: "((2699, (77, -2)), (141, (87, -2))), (-2, 169)". Could you please clarify the meaning of this "-2" and I would like to ask if you could explain the sequence with format mentioned in the paper,like the "1p queries: (e, (r,))" notation.

Thank you for your time and assistance. I look forward to hearing back from you soon.

Best regards,
Hieu

Reproduce the training of ComplEx-N3 in CQD

Hi @pminervini! Thanks for this awesome implementation of CQD.

@migalkin and I are trying to reproduce the results of CQD. For the inference part mentioned in uclnlp/cqd#7, we found that using product t-norm for all query types is enough to get good numbers. The important thing is to set --cqd-sigmoid, which is not turned on by default in the argparser. Maybe it's better to set this argument to true by default?

However, we met some troubles when reproducing the training of ComplEx-N3 in CQD. If we understand correctly, the training is carried out with the same codebase, except using only 1p task and the QA iterator. We exactly follow the hyperparameters in config.json in the checkpoint directory, and here are the beginning of our training log on FB15k-betae.

2021-12-29 12:36:19 INFO     Training average positive_sample_loss at step 0: 19.225096
2021-12-29 12:36:19 INFO     Training average negative_sample_loss at step 0: 19.225096
2021-12-29 12:36:19 INFO     Training average loss at step 0: 19.225096
...
2021-12-29 12:47:15 INFO     Training average positive_sample_loss at step 1900: 18.688958
2021-12-29 12:47:15 INFO     Training average negative_sample_loss at step 1900: 18.688958
2021-12-29 12:47:15 INFO     Training average loss at step 1900: 18.688958
2021-12-29 12:47:50 INFO     Evaluating on Valid Dataset...
2021-12-29 12:48:35 INFO     Valid 1p MRR at step 2000: 0.005592
2021-12-29 12:48:35 INFO     Valid 1p HITS1 at step 2000: 0.000137
2021-12-29 12:48:35 INFO     Valid 1p HITS3 at step 2000: 0.000373
2021-12-29 12:48:35 INFO     Valid 1p HITS10 at step 2000: 0.011451
2021-12-29 12:48:35 INFO     Valid 1p num_queries at step 2000: 59078.000000
2021-12-29 12:48:35 INFO     Valid average MRR at step 2000: 0.005592
2021-12-29 12:48:35 INFO     Valid average HITS1 at step 2000: 0.000137
2021-12-29 12:48:35 INFO     Valid average HITS3 at step 2000: 0.000373
2021-12-29 12:48:35 INFO     Valid average HITS10 at step 2000: 0.011451

This is very different from the loss and MRR in the training log of the official checkpoint.

2021-06-07 06:40:01,518 INFO     Training average positive_sample_loss at step 0: 19.225096
2021-06-07 06:40:01,518 INFO     Training average negative_sample_loss at step 0: 19.225096
2021-06-07 06:40:01,518 INFO     Training average loss at step 0: 19.225096
2021-06-07 06:45:01,820 INFO     Evaluating on Valid Dataset...
2021-06-07 06:45:49,154 INFO     Valid 1p MRR at step 500: 0.777703
2021-06-07 06:45:49,154 INFO     Valid 1p HITS1 at step 500: 0.679669
2021-06-07 06:45:49,154 INFO     Valid 1p HITS3 at step 500: 0.860898
2021-06-07 06:45:49,154 INFO     Valid 1p HITS10 at step 500: 0.931428
2021-06-07 06:45:49,154 INFO     Valid 1p num_queries at step 500: 59078.000000
2021-06-07 06:45:49,155 INFO     Valid average MRR at step 500: 0.777703
2021-06-07 06:45:49,155 INFO     Valid average HITS1 at step 500: 0.679669
2021-06-07 06:45:49,155 INFO     Valid average HITS3 at step 500: 0.860898
2021-06-07 06:45:49,155 INFO     Valid average HITS10 at step 500: 0.931428
...
2021-06-07 06:46:42,893 INFO     Training average positive_sample_loss at step 500: 8.420643
2021-06-07 06:46:42,893 INFO     Training average negative_sample_loss at step 500: 8.420643
2021-06-07 06:46:42,893 INFO     Training average loss at step 500: 8.420643

where 500iteration * 2000batch size = 1M samples. This is approximately equivalent to 2 epochs on the original FB15k dataset for knowledge graph completion, which is amazing. Our experience with the negative sampling version of ComplEx is that it doesn't converge so quickly on this dataset, though it's possible the full softmax version of ComplEx can converge faster.

Our questions are

  • Is there any insight for the QA iterator? It looks like the distribution of the QA iterator is different from the distribution of the iterator for complex query.
  • Are we missing anything important about the training of ComplEx-N3?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.