Comments (3)
In a way, it's a bug. self.hidden_dim = self.h_dim * self.num_heads
.
Maybe we should remove one of the parameters to fix the bug.
Or we move the code L232 after the best configuration setting.
In the beginning, I just want to solve the integer division problem (hidden_dim / num_heads).
Or you get other solutions?
from openhgnn.
I think we should not remove one of the parameters. I'm not sure how these two work for other models. But self.h_dim
and self.hidden_dim
actually mean different but important things in MAGNN. The former represents the hidden dimension in each head while the latter represents the sum of hidden dimension across all heads. And I see these two parameters in other models' config as well (probably those with attention mechanism). Removing them might affect a lot to some models.
Probably it'd better if we could try to differentiate the sources from which we read config arguments instead of using best config to replace original one. For example, we should read args from best_config.py
when setting --use_best_config
and from config.ini
otherwise.
I'm not sure if it's important to give hidden_dim
a clear definition, the hidden dimension in each head or the sum of hidden dimension across all heads, in multi-head attention mechanism. If we want to remove one of them, we should figure this definition out and revise model codes that confused this concept or utilized both h_dim
and hidden_dim
. But the method mentioned above might be more convenient at the moment.
from openhgnn.
I think we do not need to specify three parameters, instead, we need only two of them, as
self.hidden_dim = self.h_dim * self.num_heads
. Given two of them and the third one is determined. Considering this system, I think we only needself.hidden_dim
andself.num_heads
, as in modulepreprocess
, we needargs.hidden_dim
, so this parameter is necessary. Then in the model, we just need to replaceself.h_dim
withself.hidden_dim // self.num_heads
. This method we need to check ifself.hidden_dim
canself.num_heads
We may rename
self.h_dim
toself.head_size
like the definition indgl.gatconv
. Thehead_size
means the hidden dimension in each head. Thehidden_dim
means the output dimension of each hidden layer.
We can keep the two parameters in
config.ini
andbest_config.py
. When we usebest_config
and change it, it will not affect other parameters.
from openhgnn.
Related Issues (20)
- run HGSL model error HOT 1
- Help needed: Wanted behavior of Experiment.specific_trainerflow.get method and task/trainerflow registration HOT 2
- a running error in the link prediction task based on TransE model HOT 1
- [Doc] ReadTheDocs parameters
- RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [14328, 334]] is at version 1; expected version 0 instead HOT 7
- This demo is no longer valid HOT 1
- function load() in openhgnn/dataset/adapter.py/ HOT 1
- Run Example HOT 1
- 数据集的重复处理/预处理数据集文件缓存无效 HOT 1
- Why is the embedding of meta paths different for different nodes in acm4GTN? HOT 1
- New request for heterogeneous models
- How to support inductive predict HOT 2
- Failed to import embedding flows. HOT 1
- what's the version of numpy? An error happens. HOT 1
- how to run other tdataset with the model HOT 1
- 关于DBLP数据集精度差的问题 HOT 2
- 个别模型minibatch训练问题 HOT 1
- 无法复现HGT在HGBn-ACM数据集上的结果 HOT 2
- Segmentation fault on GTN with 3 layers HOT 1
- Obtaining metapaths and attention scores HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openhgnn.