lrjconan / lanczosnetwork Goto Github PK

Despite the dependence said on 'readme.md' page (scipy, sklearn, numpy, pytorch), I also found several other packages that require installation: pyyaml, easydict, tqdm, tensorboardx. It might be a good idea to export the python environment to yml as shown in this link [https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-from-an-environment-yml-file].

[https://github.com/lrjconan/LanczosNetwork]
"""
Dependencies
Python 3, PyTorch(1.0), numpy, scipy, sklearn
"""

In the "setup.sh" file, the program called a python script to do segment reduction based on Pytorch version and whether Cuda is available. I have acquainted myself with some knowledge on C++ and Cuda extension of Pytorch. Yet I still had difficulty understanding why we need to have the segment reduction extension. Can you explain a bit?

Thanks,
Shasha

Hi!
I am trying to reproduce the results of the paper for the citation networks.
In the paper you mention that you use the same pre-processing procedure and follow the transductive setting as in Yang et al 2016 (planetoid).
In the repository of Planetoid they provide two sets of data, one for the inductive and for the transductive version.
In the code for the paper Simplifying Graph Convolutional Networks, which you mentioned that successfully reproduces the results in Cora, the data provided is the one of the inductive version of Planetoid (ind.cora.x)
In most implementations that I have seen they use a dataset with a train (140) / valid(500)/ test (1000) split. This is also the one available in PyTorch Geometric.
The other version contains a train (140) / test (1000) split (trans.cora.x)
I would like to know which one did you use?
Thanks!

Hi @lrjconan,

Thank you for sharing this awesome code!
LanczosNet works very well on QM8, and I am working on reproducing your results on Cora dataset.
With the hyperparameters given in the appendix of the paper, I can get 80.3 accuracy with one seed (slightly higher than the reported mean in the paper). However, it seems that LanczosNet has a very high variance and often underfits (both training and test accuracy are low).

There should be some parts in the paper that I misunderstood.
Would it be possible for you to read my code and help me figure them out?
Here is the code (lanczos_net_cora.py) of LanczosNet for Cora that I modified from the QM8 model.

LanczosNet, we set the short and long diffusion scales to {1, 2, 5, 7} and {10, 20, 30} respectively.
The hidden dimension is 64 and dropout is 0.5. Lanczos step is 20. 1-layer MLP with 128 hidden
units and ReLU nonlinearity is used as the spectral filter.

Following this line, I use nfeat=1433, nhid=64, nclass=7, dropout=0.5, num_layer=2, num_eig_vec=20, spectral_filter_kind='MLP', short_diffusion_dist=[1, 2, 5, 7], long_diffusion_dist=[10, 20, 30].
The nfeat and nclass are determined by the Cora dataset.

When I printed the model, I got

LanczosNet(
  (filter): ModuleList(
    (0): Linear(in_features=11464, out_features=64, bias=True)
    (1): Linear(in_features=512, out_features=7, bias=True)
  )
  (spectral_filter): ModuleList(
    (0): Sequential(
      (0): Linear(in_features=3, out_features=128, bias=True)
      (1): ReLU()
      (2): Linear(in_features=128, out_features=3, bias=True)
    )
    (1): Sequential(
      (0): Linear(in_features=3, out_features=128, bias=True)
      (1): ReLU()
      (2): Linear(in_features=128, out_features=3, bias=True)
    )
  )
)

The L, D, V are pre-computed using the get_graph_laplacian_eigs function in your
data_helper.py script.

I use the training code from https://github.com/tkipf/pygcn.
Since there is only a single graph, the input has no batch dimension. I unsqueeze the inputs and squeeze the outputs.

Thanks,
Felix

Hi I noticed that to_graph in dataset/get_qm8_data.py doesn't use the 7th channel (distance matrix). I presume this is related to this line in your paper:

Since some models cannot leverage feature on edges easily, we use the molecule graph itself as the only input information for all models so that it is a fair comparison.

I am wondering if you have tried including the 7th channel (or if it is even valid to do so)? So to_graph would return:

...
# return atom_feat, pair_feat[:, :, :6]
return atom_feat, pair_feat[:, :, :7]

If so, what would you suggest for generating distance matrices for other molecules (not from QM8 dataset)? Is AllChem.EmbedMolecule reasonable? Thank you for making your code available!

Hi! This one is not an issue but I would appreciate if you gave your two cents.

In the "Localized Polynomial Filters" section of your paper you mention that we can build convolutional layers using the orthonormal basis in the Krylov subspace. Have you tried to implement this method? If so, can you share some results?

I did my own implementation - however I find that Chebyshev approximation performs significantly better in the standard classification tasks (cora, citeseer, pubmed). I hope you can confirm this. Propagation through the Lanczos algorithm is not that demanding though, it works fine for larger graphs.

Thank you for any help you can provide!

Hi!
Thanks for sharing your code! Would it be possible for you to mention about some preprocessing and setup rules that should be followed to test algorithms on an arbitrary dataset?

Hi,
Thanks for your sharing.

In the file

./LanczosNetwork/operators/functions/unsorted_segment_sum.py

when from operators._ext import segment_reduction
It report that ModuleNotFoundError: No module named 'operators._ext'.
Is it a file in you operators folder or a Module in python?
Thanks a lot.
Alex

I am unable to compute eigenvalues when using a scipy sparse matrix, when I am able to do so with a numpy array.

nx.convert_matrix.to_numpy_matrix(...)
nx.convert_matrix.to_scipy_sparse_matrix
get_graph_laplacian_eigs

By the way, this is very interesting work!

Great job!
I was wondering how I can do normalization for D^-1*A?