if i want to retrain the model by run train_weak.py,how and where i could get the pretrained_model:best_aug_resnet_pascal_random_smallerlr_new_strong_50_pascal_affine_resnet101_grid_loss.pth.tar and best_aug_resnet_pascal_random_smallerlr_new_strong_50_pascal_tps_resnet101_grid_loss.pth.tar

Hi,
I'd like to evaluate the model performance on Caltech101 dataset but the link http://www.di.ens.fr/willow/research/weakalign/other_resources/test_pairs_caltech_with_category.csv
has failed.
Can you upload a new one?
Thanks

trying out demo.py, getting an error:

Traceback (most recent call last):
  File "demo.py", line 8, in <module>
    from model.cnn_geometric_model import CNNGeometric, TwoStageCNNGeometric
  File "/Users/tkramer/projects/weakalign/model/cnn_geometric_model.py", line 9, in <module>
    from util.conv4d import Conv4d, Linear4d
ModuleNotFoundError: No module named 'util.conv4d'

Thanks for your sharing the code for this project. I just have a question about the implementation of the affTpsTnf function in Demo.py.

def affTpsTnf(source_image, theta_aff, theta_aff_tps, use_cuda=use_cuda):
    tpstnf = GeometricTnf(geometric_model = 'tps',use_cuda=use_cuda)
    sampling_grid = tpstnf(image_batch=source_image,
                           theta_batch=theta_aff_tps,
                           return_sampling_grid=True)[1]
    X = sampling_grid[:,:,:,0].unsqueeze(3)
    Y = sampling_grid[:,:,:,1].unsqueeze(3)
    Xp = X*theta_aff[:,0].unsqueeze(1).unsqueeze(2)+Y*theta_aff[:,1].unsqueeze(1).unsqueeze(2)+theta_aff[:,2].unsqueeze(1).unsqueeze(2)
    Yp = X*theta_aff[:,3].unsqueeze(1).unsqueeze(2)+Y*theta_aff[:,4].unsqueeze(1).unsqueeze(2)+theta_aff[:,5].unsqueeze(1).unsqueeze(2)
    sg = torch.cat((Xp,Yp),3)
    warped_image_batch = F.grid_sample(source_image, sg)

    return warped_image_batch

In this implementation, you first use the TPS and then the affine transformation. However, when training the network, the model adopt the affine transformation before the TPS operation. I'm just a little bit confused about it and is it right?

I have checekd the CNNGeo in your another repo.

theta_aff_tps = model_tps({'source_image': warped_image_aff, 'target_image': batch['target_image']})
warped_image_aff_tps = tpsTnf(warped_image_aff, theta_aff_tps)

In this implementation, you first do the affine transformation and then follow the TPS transformation. Are there any differences between these two implementions?

Looking forward to your reply.

Thanks for the amazing work.

Trained models can't be accessed (lands at "Forbidden - You don't have permission to access this resource"). Same is true for other repos with models stored at https://www.di.ens.fr/willow/.

Any alternative links by any chance?

Hello,here is my question:
If I want to match one image in the different affine transforms, should I use the CNNGeometric or this weakalign? which one is better for my needs? Thank you very much.

I tried to use train_weak_pf_pascal.sh, however, "best_aug_resnet_pascal_random_smallerlr_new_strong_50_pascal_affine_resnet101_grid_loss.pth.tar" is not found, how can i find it ?

I downloaded weakalign_resnet101_affine_tps.pth.tar and cnngeo_resnet101_affine.pth.tar, and both of them cannot be unzipped (or un-tar) by WinRar in windows or using "tar -xvf ..." in Linux, both of them give a message that the file is not in Tar format or damaged, could you please check why it happened?

Hi great work, thank you for the code

can you please specify exactly the command you used to train your combined aff+TPS model...
because I want to recreate your results
thanks

Hi, I test cnngeo_vgg16_tps.pth.tar on the PF-WILLOW dataset, the result (PCK 51.28%) is different with your CVPR 2017 paper (PCK 42.8%). If the two models are trained on the same dataset?

Hi, could you help me with how to obtain experimental results on TSS, please? In the code, only flow results are written as files, no evaluation results of Table 4 in CVPR 2018 paper are shown. Thank you for the help.

Traceback (most recent call last):
File "/home/weakalign-master/demo.py", line 77, in
model.FeatureExtraction.state_dict()[name].copy_(checkpoint['state_dict']['FeatureExtraction.' + name])
KeyError: 'FeatureExtraction.model.1.num_batches_tracked'

I tried your code, but there was KeyError like above
OrderedDict "checkpoint['state_dict']['FeatureExtraction.model.1.num_batches_tracked']" does not exist
I'd appreciate it if you could check for the error.

Hello,
Do you consider choosing a open source license for potential futur works based on your promising approach ?
Thanks!

hello,when I try to run your code , i get the transformation model successfully. When reading your paper, I see that you give some examples to show feature points matching results , including inliers and outliers matching results.When I read your code, I have not found the relating function .May you really do it in your code ,but I just have not found it .So I am sorry to bother you to ask for this question .THANKS !

Hello,

I'm trying to run demo.py but keep running into this error no matter which data set I use, perhaps I did something wrong with the setup? Any help would be appreciated. Thanks!

The error:

Traceback (most recent call last):
  File "C:\Users\trist\OneDrive\Documents\weakalign\demo.py", line 72, in <module>
    model.FeatureExtraction.state_dict()[name].copy_(checkpoint['state_dict']['FeatureExtraction.' + name])
KeyError: 'FeatureExtraction.model.1.num_batches_tracked'

Hi, Thanks for the nice package and code. I had a few issues in runnig the code on PyTorch 0.4, especially in reading the model. I saw a few similar open issues which people sugessted to change the verison to PyTorch 0.2. Since Most of the recent packages are based on PyTorch 0.4+, and I wasn't intrested in use the conda solution or downgrading, I spent time to find a way to run the code. This issue is just for sharing the location that you may need to change for the recent version of the PyTorch:

1- Loading the pre-trained model issue:

OrderedDict "checkpoint['state_dict']['FeatureExtraction.model.1.num_batches_tracked']" does not exist I'd appreciate it if you could check for the error.

To solve this you need to find the comman names between this pretrained checkpoint and TwoStageCNNGeometric model namespaces.

    if model_aff_tps_path != '':
        checkpoint = torch.load(model_aff_tps_path, map_location=lambda storage, loc: storage)
        checkpoint['state_dict'] = OrderedDict(
            [(k.replace('vgg', 'model'), v) for k, v in checkpoint['state_dict'].items()])

        for name, param in model.FeatureExtraction.state_dict().items():
            if 'FeatureExtraction.' + name in checkpoint['state_dict']:
                model.FeatureExtraction.state_dict()[name].copy_(checkpoint['state_dict']['FeatureExtraction.' + name])
        for name, param in model.FeatureRegression.state_dict().items():
            if 'FeatureRegression.' + name in checkpoint['state_dict']:
                model.FeatureRegression.state_dict()[name].copy_(checkpoint['state_dict']['FeatureRegression.' + name])
        for name, param in model.FeatureRegression2.state_dict().items():
            if 'FeatureRegression2.' + name in checkpoint['state_dict']:
                model.FeatureRegression2.state_dict()[name].copy_(checkpoint['state_dict']['FeatureRegression2.' + name])

The other optimum way is to add a more pythonic statement when you are generating the checkpoint and make a OrderedDic, something like:

 [(k.replace('vgg', 'model'), v) for k, v in checkpoint['state_dict'].items() if v in model.FeatureExtraction.state_dict()])

but I it doesn't work like this definitly and I couldn't find an optimum way anyway, so I ended up to add the explicit if statements.

2- The second issue is happening later on, in preprocess_image(), normalize_image() in ./image/normalization.py, line 38

if isinstance(image,torch.autograd.variable.Variable):
....

The fact is in the classes Tensor and Variable got merged in newer version of PyTorch, so there is no need to check if image is a Variale type and so on.

So, you can easily replace this wholeline by "else:"

Hope this would be helpful for others too.

Hi, thanks for sharing the code. But I noticed that during the evaluation process, in the two-stage case, the same pretrained affine and tps models are loaded as evaluating affine / tps model in a one-stage manner. As far as I know, the tps model in two-stage regime should be kind of different from the one in one-stage, for the image A fed to this model has been warped with an affine transformation. Would you please give a clarification about that? @ignacio-rocco

Hello,

Is there any file in the code or otherwise a guide on how to go about constructing a data set that can be read in for training/eval from .jpg images? I've inspected the structure of the provided datasets and I don't think it will be feasible to manually imitate it. Any help would be greatly appreciated.

Thanks!

Hi,
First, thank you for sharing your work.

For the line of code attached, why are you passing "corr_aff" for the match?
Shouldn't it be "corr_aff_tps"?

"mask_id" in the loss computation code goes through transformation up to tps.
Wouldn't it be more appropriate to masking it with "corr_aff_tps"?

Thanks.