Leaderboard for heterogeneous graphs is missing about cogdl HOT 4 OPEN

Hi @amblee0306 ,

You can find the cogdl leaderboard from this link. If you want to see the original repo of Simple-HGN, please check this link

from cogdl.

Hi @cenyk1230

Thanks for your reply, the leaderboard stated that the models were evaluated using macro-f1 scores, can I check how can I change the code such that the macro-f1 scores are returned? Currently when I run the simple-hgn from cogDL, I believe the test_acc is being utilized instead.

from cogdl.

Hi @amblee0306 ,

It seems that the results in the leaderboard should be micro-f1 scores (the same as accuracy in the multi-class setting) rather than macro-f1.

from cogdl.

Hi @cenyk1230

Yes indeed. I would like to get the macro-f1 scores as well. Do you have any advice on how that is possible? Also, the score on the leaderboard, is it only for one instance or it refers to the average micro-f1 score? Cause I ran 10 instances and the average was lower than what was mentioned on the leaderboard.

from cogdl.