yrcong / nodis Goto Github PK

As the guideline says, models for sgcls and predcls are trained from checkpoints/vgdet/vg-24.tar. Model for sgdet may need to be trained based on the former models, but which one to use? Or maybe model for sgcls should be trained based on model for predcls?

Would mind giving some instruction on implementing NODIS with GCN rather than LSTM? It seems that The existing file "ODE.py" in the project only constructs ODE blocks based on LSTM. But as GCN need an adjacent matrix in addition, it seems not possible to simply replace LSTM with GCN ("odeint" function seems to only accept one input tensor). I would appreciate if any suggestion is given.

I followed the training guidance and use the command
python train_rels.py -m sgcls -order random -b 6 -p 100 -lr 1e-4 -ngpu 1 -ckpt checkpoints/vgdet/vg-24.tar -save_dir checkpoints/ -nepoch 20
to train the model. But the performance seems to be lower than reported. So I kept refining the model with the following command:
python train_rels.py -m sgcls -order random -b 6 -p 100 -lr 1e-4 -ngpu 1 -ckpt checkpoints/vgrel-19.tar -save_dir checkpoints/ -nepoch 11
I got the checkpoint vgrel-30.tar, which resulted in a performance like the following (I have also done a similar procedure for PREDCLS ):

Although the results are close, I still want to make a more precise reproduction. Could you please give some advice on it?

Following the decribtion in the paper, I add a new ODE function:
class odeFunc3(nn.Module):

def __init__(self, use_cuda):
    super(odeFunc3, self).__init__()
    self.use_cuda = use_cuda
    self.linear = nn.Linear(1024, 1024)
    self.hidden_state = None

def init_hidden(self, batch_size):
    return None

def forward(self, t, x):
    x = self.linear(x)
    return x

and replace the two odeFunc used in NODIS.py with it. Then I train the PREDCLS model with:
python models/train_rels.py -m predcls -order random -b 6 -p 100 -lr 1e-4 -ngpu 1 -ckpt checkpoints/vgdet/vg-24.tar -save_dir checkpoints/ -nepoch 20
But when I test the resulting model with:
python models/eval_rels.py -m predcls -order random -b 6 -p 100 -lr 1e-3 -ngpu 1 -test -ckpt checkpoints/vgrel-19-0.6350.tar -nepoch 50
I only got 0.658502 on R@100 for the PREDCLS task.
Did I miss anything?