petarv- / dgi Goto Github PK

Deep Graph Infomax (https://arxiv.org/abs/1809.10341)

License: MIT License

Python 100.00%

deep-graph-infomax pytorch neural-networks graph-convolutional-networks python unsupervised-learning unsupervised-node-embedding

dgi's Introduction

DGI

Deep Graph Infomax (Veličković et al., ICLR 2019): https://arxiv.org/abs/1809.10341

Overview

Here we provide an implementation of Deep Graph Infomax (DGI) in PyTorch, along with a minimal execution example (on the Cora dataset). The repository is organised as follows:

data/ contains the necessary dataset files for Cora;
models/ contains the implementation of the DGI pipeline (dgi.py) and our logistic regressor (logreg.py);
layers/ contains the implementation of a GCN layer (gcn.py), the averaging readout (readout.py), and the bilinear discriminator (discriminator.py);
utils/ contains the necessary processing subroutines (process.py).

Finally, execute.py puts all of the above together and may be used to execute a full training run on Cora.

Reference

If you make advantage of DGI in your research, please cite the following in your manuscript:

@inproceedings{
velickovic2018deep,
title="{Deep Graph Infomax}",
author={Petar Veli{\v{c}}kovi{\'{c}} and William Fedus and William L. Hamilton and Pietro Li{\`{o}} and Yoshua Bengio and R Devon Hjelm},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=rklz9iAcKQ},
}

License

MIT

dgi's People

Contributors

Stargazers

Watchers

Forkers

awesome-archive xuchensjtu qss2012 deep-mining-swang bousejin yflyzhang gear dendisuhubdy naihemeng kckishan hhh920406 jerry185 liqimai liangxun bruinxiong tanxiaobing-hl zizai xuanheiiis chenxingqiang m397026474 hoangdzung masoudbimar ashutosh-adhikari haopengzhang96 chenxinst chor-nyan yinhongyh snazari liun-online sienna13 amber0111 hekpomah junting98 ccfbupt jafermarq matthew-tech sean0719 5l1v3r1 lemonqc mehdichelh lizhong2613 milkigit realcatking mess-clarifai tyut2018 chenchengkuan zanghyu thilinicooray yuanwanglll aabbccgithub struggleallen carrotyq hanbei969 deng-fankang chaoyan1037 lzw27 mldl daokunzhang biayangqi changjiale3 baobunuo cognoscentai zxcayumi kevinchow666 pc2005 butterluo scncdba tony109060581 mtang724 jerry2398 utoku lukeng105 zm235575 saqibmamoon gombessa1938 tianyuzelin anthonyxn whxhx jiehu-cv ljeub harry-zhou mpanpan wangkeao icloudsong xikakera lgalke blvren liatat ljxw88 zzysh12345 sohailkhanmarwat devvrit fatcatzf lyyf2002 ftahabi chenchenhaha chaoyingyang g-taxonomy-workgroup duykhuongnguyen spocksdad

dgi's Issues

Codes on reddit dataset

Great work! I really enjoy reading the paper.

However, will the codes to replicate the reported performance on reddit be released? If so, is there a planned schedule?

Thank you!

About the meaning of learned features

Hi, I was wondering that the learned representations tend to conserve their unique information or common information. Maximizing Mutual information between patch and summary vector is to find more information between them, but the discriminator wants to distinguish the samples. So, I am confused.

objective function in the paper?

great paper!
I have a question about the objective function, is it same as cross_enpty?
looking forward to your reply!

Why do we need to calculate for expectation before sum

   Hello, I've read your wonderful paper published on ICLR, and I'd like to consult you some questions. 
   The two summation symbols in the objective function sum the positive and negative samples and find the average value. Why do you need to calculate the expectation before sum?
   thank you!

Mean accuracy on fixed representation

HI, great work ! But I have a question regarding how to repeat the experiments.
While getting the mean accuracy, you used the fixed representation and only repeated the logistic regression part. Shall we repeat the whole pretext training + downstream task altogether ?
Or is there any reference for the reason of doing so ?
Many thanks

Script for inductive training

Hi,

Can you please share your code for training and evaluating PPI dataset as well?

Thanks

sigmoid function is missing in layers/discriminator.py

It seems that the sigmoid function is missing in layers/discriminator.py[line30].
Explained in your paper, the logistic sigmoid nonlinearity is used to convert scores into probabilities of (h_i, s) being a positive example.

DATA

What is the kind of your dataset(cora).
How can i make my graph like this.
thank you very much.

Out of Memory on Pubmed Dataset

I tried to run the released execute.py on Pubmed. However, it seems that it takes 19.25 GB during back propagation.

Is this the correct behavior? Is there any solution to bypass this problem and replicate the paper reported number?

The time cost on Reddit

Hi Petar,
Can you provide the time cost on Reddit and PPI datasets?
Now I am doing some work to scale up Graph Auto Encoder, and I wonder how efficient is this method?

Thanks

Help！

I saw the DGI implementation which only contains Cora dataset on your github

I was wondering whether you could kindly share your DGI implementation which contains Reddit and PPI dataset!

Thank you very much！

Could anybody explain what are x，tx，allx，y，ty，ally?

yes this is a brief cora experiment,
but the data is so different from original data set cora.
could anybody explain what they mean?
thanks a lot!

If I want to add features for my edges, can I simply replace the encoder network with a network that supports edge_attr?

Hi Petar,
I have read your idea abot Deep Graph Infomax.
While right now something really puzzles me: if I want to add features for my edges and I still want to do unsupervised or self-supervised learning. Can I just simply change the encoder and remain all the other function unchanged? Such as corruption function or readout function?

The code for Reddit！HELP！

Question about PPI dataset

Hi,

Thank you for making the code available. I would like to ask a question about a remark you made in your paper about the PPI dataset. On page 8, paragraph "Inductive learning on multiple graphs", you noted that 40% of the node has all-zero feature vector. However, the feature vectors loaded using GraphSAGE (http://snap.stanford.edu/graphsage/) is dense. Did you use a different set of feature vectors or a different PPI dataset?

Thank you for your time! If I misunderstood something, please kindly point out my mistake.

Edit: Sorry I made a mistake when I check the feature vectors. It was indeed 42% all zeros.

Error in AvgReadout

Hi, I think there's an error in AvgReadout with a mask. The mask summation should be performed along the first dimension only.

DGI/layers/readout.py

Line 15 in 0afce4e

return torch.sum(seq * msk, 1) / torch.sum(msk)

It is
return torch.sum(seq * msk, 1) / torch.sum(msk)
but should be
return torch.sum(seq * msk, 1) / torch.sum(msk, 1) # Note the dimension here.

thanks