Comments (5)
Hi @larry2020626 ,
Glad to help you reproduce the result. Please kindly provide the following information:
- Which model are you evaluating? (moco/e2e, downloaded/pretrained);
- Your device (cpu/cuda);
- Your pytorch/dgl version;
- Your obtained result on airport.
In addition, did you try other datasets/tasks? Please feel free to provide more datapoints/screenshots.
Thanks!
from gcc.
Thanks very much for your reply!
I adopt E2E model, running on CUDA 10.0
My Device:
40 Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz
3b:00.0 3D controller: NVIDIA Corporation Device 1eb8 (rev a1)
5e:00.0 3D controller: NVIDIA Corporation Device 1eb8 (rev a1)
pytorch version: 1.4.0+cu100
dgl version '0.4.3post2'
current result is {'Micro-F1': 0.6218}
here is what I do:
- to download small.bin pretrain dataset
python scripts/download.py --url https://cloud.tsinghua.edu.cn/f/b37eed70207c468ba367/?dl=1 --path data --fname small.bin
2.to pretrain:
bash scripts/pretrain.sh 0 --batch-size 256
3.For downstream tasks:
python scripts/download.py --url https://cloud.tsinghua.edu.cn/f/2535437e896c4b73b6bb/?dl=1 --path data --fname downstream.tar.gz
bash scripts/generate.sh 0 Pretrain_moco_False_dgl_gin_layer_5_lr_0.005_decay_1e-05_bsz_256_hid_64_samples_2000_nce_t_0.07_nce_k_32_rw_hops_256_restart_prob_0.8_aug_1st_ft_False_deg_16_pos_32_momentum_0.999 usa_airport
bash scripts/node_classification/ours.sh saved/Pretrain_moco_False_dgl_gin_layer_5_lr_0.005_decay_1e-05_bsz_256_hid_64_samples_2000_nce_t_0.07_nce_k_32_rw_hops_256_restart_prob_0.8_aug_1st_ft_False_deg_16_pos_32_momentum_0.999 64 usa_airport
Thanks!
And I will try to use downloaded models to evaluate on usa-airports dataset to check whether it is because my pre-training effect is not good enough.
from gcc.
Hi @larry2020626 ,
Thanks for your reply. The main reason is that your experiments correspond to the GCC (E2E, freeze) model, while "68.3" is the result of GCC (E2E, full). For the difference, please see the Freezing vs. full fine-tuning paragraph at the beginning of page 6.
To obtain the "full" (finetuning) result, please run python train.py --exp FT --model-path saved --tb-path tensorboard --tb-freq 5 --gpu 0 --dataset usa_airport --finetune --epochs 30 --resume saved/Pretrain_moco_False_dgl_gin_layer_5_lr_0.005_decay_1e-05_bsz_256_hid_64_samples_2000_nce_t_0.07_nce_k_32_rw_hops_256_restart_prob_0.8_aug_1st_ft_False_deg_16_pos_32_momentum_0.999/current.pth --cv
. This will start finetuning the model on usa_airport on 10 splits, and the final mean/std of the accuracy will be reported. The result on my side is 66.7±4.1.
Besides, you can use batch size 1024 in paper instead of 256 but that will take even longer for pretraining. In that case, the result is 68.3±2.8
.
from gcc.
Thanks!@qibinc , really appreciate your help.
from gcc.
Hi Larry, I'm glad it worked for you. Feel free to raise more issues if you encounter other problems. Closing this.
from gcc.
Related Issues (20)
- it seems dgl-0.4.1 cannot work properly HOT 1
- How to make data?
- Running experiments completely on CPU HOT 1
- An error when finetuning on graph classification dataset HOT 3
- Questions about pretraining subgraphs
- Possibly redundant BatchNorm layer? HOT 1
- Can't do node_classification tasks on panther datasets HOT 1
- torch.util.data.IterableDataset HOT 1
- About downstream datasets HOT 2
- When I run the algorithm get an error: "OSError: libcudart.so.10.0: cannot open shared object file: No such file or directory" HOT 3
- finetune HOT 2
- x2dgl.py
- kdd17数据集中的.txt.lpm.lscc
- Please help resolve the problem when the code was running on CUDA11.1 , DGL 0.7, HOT 1
- Regarding the choice of subgraph enhancement method.
- how to unzip small.bin HOT 1
- no edge label for the datasets?
- Can`t running this code with low equipments HOT 5
- In generating phase, the necessarity of increasing batch size to the length of dataset. HOT 3
- an error when running x2dgl.py
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gcc.