I am used to debug using pycharm and I found it hard to debug using jupyter notebook; for example if I want to step into some code, I use pdb; maybe I haven't found the tricks to debug using jupyter. So can I debug the code using pycharm?

Hi, the way you implement siamese network is like that:

output1,output2 = model(x0, x1)
loss = criterion (output1,output2)

And here is the another way:

output1 = model.forward_once(x0)
output2 = model.forward_once(x1)
loss = criterion (output1,output2)

Are they different?

Is it normal for testing to take so long? And so far its only outputted one testing result.

Sorry, I am a beginner.
I could not find the results after training.
It makes me feel that actually testing and training are two different things.
Thank you very much

hey your work is quite good and i tried implementing it and was gud to see it worked. i need some help regarding the test directory, i want to make this work for webcam, the test image should be taken from webcam and the similarity should be tested with the image take from the webcam. but i can't understand how to do that. plz help me to achieve it.

Thanks for sharing your project. I just met a small question when reading your code. For

class SiameseNetworkDataset(Dataset):

def __getitem__(self,index):
    img0_tuple = random.choice(self.imageFolderDataset.imgs)
    #we need to make sure approx 50% of images are in the same class
    should_get_same_class = random.randint(0,1) 
    if should_get_same_class:
        while True:
            #keep looping till the same class image is found
            img1_tuple = random.choice(self.imageFolderDataset.imgs) 
            if img0_tuple[1]==img1_tuple[1]:
                break
    else:
        while True:
            #keep looping till a different class image is found
            
            img1_tuple = random.choice(self.imageFolderDataset.imgs) 
            if img0_tuple[1] !=img1_tuple[1]:
                break
        img1_tuple = random.choice(self.imageFolderDataset.imgs)

when should_get_same_class equals to False, the coding keeps running until it finds img1_tuple that has different label as img0_tuple, but why do you use img1_tuple = random.choice(self.imageFolderDataset.imgs) again in last row?

Thank you very much.

Thanks for putting this up for others to look at.

I noticed in the dataset code that if the random.randint(0,1) ends up being 0, the code that runs is img1_tuple = random.choice(self.imageFolderDataset.imgs). This can (though low probability) choose a img from the same folder. This would mis-label the img pairs as not being the same, but indeed they are.

while trying to train code, I encountered the following error:

'''TypeError: mul received an invalid combination of arguments - got (Variable), but expected one of:

• (float value)
  didn't match because some of the arguments have invalid types: (Variable)
• (torch.FloatTensor other)
  didn't match because some of the arguments have invalid types: (Variable)'''

I'm a bit stuck at this point, I was just running the ipython implementation.

This is very cool. Thank you for this.

I'm interested to get the output as a table: image1.jpg, image2.jpg, dissimilarity score.

I understand that this is not something the dataloader can do, so I'd have to modify, as in eg: https://discuss.pytorch.org/t/dataloader-filenames-in-each-batch/4212

...but I'm not having much success. Wondered if there was something blindingly obvious that I'm just missing.

Thanks

Hi,
I downloaded this repo and I think the only modification I made to the code was to change the first line in In [15] from net = SiameseNetwork().cuda() to net = SiameseNetwork()#.cuda(). I made this change because I don't have a dedicated Nvidia GPU on my laptop.

I'm getting the error

TypeError: unsupported operand type(s) for /: 'tuple' and 'int'

triggered by this line: for i, data in enumerate(train_dataloader,0):

I haven't been able to figure out a solution. Any help is much appreciated.

Hi all, I have tried the notebook removing the cuda part since I have no gpu. Two things that I noticed, 1) loss gets below 0.2 after 10 epoch (which is way better than what is reported using gpu), and 2) in testing I get dissimilarity more than 1 (I was hoping to get a number between 0 and 1). I am wondering is there anything wrong here?

im using this code with street image dataset it's just keep on running the loop, whats the problem ?

folder_dataset_test = dset.ImageFolder(root=Config.testing_dir)
siamese_dataset = SiameseNetworkDataset(imageFolderDataset=folder_dataset_test,
transform=transforms.Compose([transforms.Resize((100,100)),
transforms.ToTensor()
])
,should_invert=False)

test_dataloader = DataLoader(siamese_dataset,num_workers=6,batch_size=1,shuffle=True)
dataiter = iter(test_dataloader)
x0,, = next(dataiter)

for i in range(10):
_,x1,label2 = next(dataiter)
concatenated = torch.cat((x0,x1),0)

output1,output2 = net(Variable(x0).cuda(),Variable(x1).cuda())
euclidean_distance = F.pairwise_distance(output1, output2)
imshow(torchvision.utils.make_grid(concatenated),'Dissimilarity: {:.2f}'.format(euclidean_distance.item()))

Why is the training process being done after conversion to Grayscale? Wouldn't it mean that you don't utilize all features?
@harveyslash

Hi, I was wondering why do you apply Dropout2D (which drops a colour channel given certain probability) in the network. As far as I know, you convert the images to single channel greyscale before feeding these to the network, which means that Dropout2D is getting rid of the single channel you have 20% of the times.

Also, what is the difference when it comes to learning improvement when using Reflective Padding?

Thanks, great tutorial and repo by the way.

Dani

Hi,

I couldnt find the rquirements.txt in the github link you have provided
Kindly update it.

Hi,

What should be the format of files in "./data/faces/training/"?
I downloaded the AT&T dataset inside this folder but i still get the following error:

RuntimeError: invalid argument 1: must be strictly positive at /pytorch/torch/lib/TH/generic/THTensorMath.c:2033

it is because len(siamese_dataset)=0. It makes me think that I am not using the correct data file format.

Hello,

I would like to integrate some classic CNN, e.g. AlexNet, VGG16, into this project. Take AlexNet as an example, I discard the last layer of original AlexNet, and use the second last layer to generate a 1*4096 feature vector.

But after the implementation, I found it does not work. The dissimilarity scores for almost all pairs predicted by the model equal zero. Does anyone have idea about that? Or is it possible to integrate those classic and deeper CNN into this project?

@Fanshia Hi, Fanshia, i found a question in your code, it's about contrastive loss function.
euclidean_distance = F.pairwise_distance(output1, output2)

in PyTorch Docs
def pairwise_distance(x1, x2, p=2., eps=1e-6, keepdim=False):
https://pytorch.org/docs/stable/_modules/torch/nn/functional.html#pairwise_distance

so, "euclidean_distance" maybe it should be like this euclidean_distance = F.pairwise_distance(output1, output2, keepdim = True)
that's why the loss curve is instability, and test performance is bad, If you modify it like this, you will found the performance is well! To be honest, this problem has also plagued me for a long time. But anyway, thank you for your works, it's a very good tutorial.

Hello ~

I am a learner of pytorch , I run your code in my pc , but it report BrokenPipeError: [Errno 32] Broken pipe ,and my os is Win10 ,Pytorch is 0.4 and Jupyter is 1.0

the details are as follow

BrokenPipeError Traceback (most recent call last)
in ()
1 vis_dataloader = DataLoader(siamese_dataset,shuffle=True,num_workers=8,batch_size=8)
----> 2 dataiter = iter(vis_dataloader)
3
4
5 example_batch = next(dataiter)

D:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in iter(self)
449
450 def iter(self):
--> 451 return _DataLoaderIter(self)
452
453 def len(self):

D:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in init(self, loader)
237 for w in self.workers:
238 w.daemon = True # ensure that the worker exits on process exit
--> 239 w.start()
240
241 _update_worker_pids(id(self), tuple(w.pid for w in self.workers))

D:\Anaconda3\lib\multiprocessing\process.py in start(self)
103 'daemonic processes are not allowed to have children'
104 _cleanup()
--> 105 self._popen = self._Popen(self)
106 self._sentinel = self._popen.sentinel
107 # Avoid a refcycle if the target function holds an indirect

D:\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
221 @staticmethod
222 def _Popen(process_obj):
--> 223 return _default_context.get_context().Process._Popen(process_obj)
224
225 class DefaultContext(BaseContext):

D:\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
320 def _Popen(process_obj):
321 from .popen_spawn_win32 import Popen
--> 322 return Popen(process_obj)
323
324 class SpawnContext(BaseContext):

D:\Anaconda3\lib\multiprocessing\popen_spawn_win32.py in init(self, process_obj)
63 try:
64 reduction.dump(prep_data, to_child)
---> 65 reduction.dump(process_obj, to_child)
66 finally:
67 set_spawning_popen(None)

D:\Anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
58 def dump(obj, file, protocol=None):
59 '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60 ForkingPickler(file, protocol).dump(obj)
61
62 #

BrokenPipeError: [Errno 32] Broken pipe

Hi, i'm working on this siamese-networks with resnet50, and a new big dataset(the dataset is correct). Now, i have a problem, my loss function converges badly. my self.margin = 1.0, to be honest, i don't know how to set this, Is this parameter important? Do you have any advice? thx!

I dont understand how to set a threshold, with which we can say if its the same person or not. Say if distance is > 0.5 apart, its different person ... less than 0.5 they are the same person.

Thanks in advance :)

Hlo there, I’m new to python / pytorch. I have one folder(Folder1) contains 100 images of different of different person, but another folder(Folder2) contains 100 image different person (but same persons as in Folder1).

_folder1/p1.jpg
folder1/p2.jpg
folder1/p3.jpg
.
.
.folder1/p100.jpg

folder2/p1.jpg
folder2/p2.jpg
folder2/p3.jpg
.
.
.folder2/p100.jpg_
Same name indicate same person. I’m tring to modify this code, to train my system. But I don’t have any idea. Plz help me.
Thnks

In your code <= 0.5 means the persons are the same...How did you set it (coz when I ran your same code the resulting floating euclidean distances were a bit greater)... I was looking for its cosine similarity (limiting range : min 0 to max 1) how can we set it?

pip install -r requirements.txt
Collecting bleach==2.0.0 (from -r requirements.txt (line 1))
Downloading bleach-2.0.0-py2.py3-none-any.whl
Collecting decorator==4.1.1 (from -r requirements.txt (line 2))
Using cached decorator-4.1.1-py2.py3-none-any.whl
Collecting entrypoints==0.2.3 (from -r requirements.txt (line 3))
Using cached entrypoints-0.2.3-py2.py3-none-any.whl
Collecting html5lib==0.999999999 (from -r requirements.txt (line 4))
Using cached html5lib-0.999999999-py2.py3-none-any.whl
Collecting ipykernel==4.6.1 (from -r requirements.txt (line 5))
Using cached ipykernel-4.6.1-py2-none-any.whl
Collecting ipython==6.1.0 (from -r requirements.txt (line 6))
Could not find a version that satisfies the requirement ipython==6.1.0 (from -r requirements.txt (line 6)) (from versions: 0.10, 0.10.1, 0.10.2, 0.11, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 1.0.0, 1.1.0, 1.2.0, 1.2.1, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.3.1, 2.4.0, 2.4.1, 3.0.0, 3.1.0, 3.2.0, 3.2.1, 3.2.2, 3.2.3, 4.0.0b1, 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.1.0rc1, 4.1.0rc2, 4.1.0, 4.1.1, 4.1.2, 4.2.0, 4.2.1, 5.0.0b1, 5.0.0b2, 5.0.0b3, 5.0.0b4, 5.0.0rc1, 5.0.0, 5.1.0, 5.2.0, 5.2.1, 5.2.2, 5.3.0, 5.4.0, 5.4.1, 5.5.0)
No matching distribution found for ipython==6.1.0 (from -r requirements.txt (line 6))

In the original notebook, it states:

"The top row and the bottom row of any column is one pair. The 0s and 1s correspond to the column of the image. 0 indiciates dissimilar, and 1 indicates similar."

However, this contradicts both the blogpost and the actual output:

"The Siamese Network dataset generates a pair of images , along with their similarity label (0 if genuine, 1 if imposter)."

The PR that I'm about to submit fixes this.

Instead of

def forward(self, output1, output2, label):
        euclidean_distance = F.pairwise_distance(output1, output2)
        loss_contrastive = torch.mean((1-label) * torch.pow(euclidean_distance, 2) +
                                      (label) * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))

        return loss_contrastive

The loss function should be this right?:

def forward(self, output1, output2, label):
        euclidean_distance = F.pairwise_distance(output1, output2)
        loss_contrastive = 0.5*((1-label) * torch.pow(euclidean_distance, 2) +
                                      (label) * torch.pow(torch.clamp(self.margin - euclidean_distance, min=0.0), 2))

        return loss_contrastive.mean()

when I run following segment of code from your Siamese network code I got this error:
Anaconda Environment Python 3.6 windows 10 system:

vis_dataloader = DataLoader(siamese_dataset,
shuffle=True,
num_workers=8,
batch_size=8)
dataiter = iter(vis_dataloader)

example_batch = next(dataiter)
concatenated = torch.cat((example_batch[0],example_batch[1]),0)
imshow(torchvision.utils.make_grid(concatenated))
print(example_batch[2].numpy())

BrokenPipeError Traceback (most recent call last)
in ()
3 num_workers=8,
4 batch_size=8)
----> 5 dataiter = iter(vis_dataloader)
6
7

C:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in iter(self)
817
818 def iter(self):
--> 819 return _DataLoaderIter(self)
820
821 def len(self):

C:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in init(self, loader)
558 # before it starts, and del tries to join but will get:
559 # AssertionError: can only join a started process.
--> 560 w.start()
561 self.index_queues.append(index_queue)
562 self.workers.append(w)

C:\Anaconda3\lib\multiprocessing\process.py in start(self)
103 'daemonic processes are not allowed to have children'
104 _cleanup()
--> 105 self._popen = self._Popen(self)
106 self._sentinel = self._popen.sentinel
107 _children.add(self)

C:\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
221 @staticmethod
222 def _Popen(process_obj):
--> 223 return _default_context.get_context().Process._Popen(process_obj)
224
225 class DefaultContext(BaseContext):

C:\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
320 def _Popen(process_obj):
321 from .popen_spawn_win32 import Popen
--> 322 return Popen(process_obj)
323
324 class SpawnContext(BaseContext):

C:\Anaconda3\lib\multiprocessing\popen_spawn_win32.py in init(self, process_obj)
63 try:
64 reduction.dump(prep_data, to_child)
---> 65 reduction.dump(process_obj, to_child)
66 finally:
67 set_spawning_popen(None)

C:\Anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
58 def dump(obj, file, protocol=None):
59 '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60 ForkingPickler(file, protocol).dump(obj)
61
62 #

BrokenPipeError: [Errno 32] Broken pipe

The last sentence nn.Linear(500, 5)， why 5 ?

This part is working with the faces dataset only because the number of classes is big enough, but it will not generalize well to dataset where the number of classes is small (e.g. 2).

Basically, the else branch should also have a while True: with the opposite logic. Since you don't have that, the else branch may indeed choose an img1_tuple of the same class of img1_tuple, with a probability of 1/n_classes. So, if you have 40 classes, it's very unlikely that that will happen, thus your sample works fine, but if you have 2 classes that will happen 50% of the time, which would mean that instead of using 50%-50% you'd be using 75%-25%.

     should_get_same_class = random.randint(0, 1)
     if should_get_same_class:
         while True:
             # keep looping till the same class image is found
             img1_tuple = random.choice(self.imageFolderDataset.imgs)
             if img0_tuple[1] == img1_tuple[1]:
                 break
     else:
         img1_tuple = random.choice(self.imageFolderDataset.imgs)

I know it's only an example, but that part tricked me for a while, so I thought it was worth mentioning.

The loss is wrong due to shape inconsistent.

Thank you for your code, I change the code of pytorch to paddlepaddle, and deploy it on Baidu free server to run online. I have tested both your network and RESNET network, and found that this method is not very effective. My test is to use a graph of each category as the database, and then use a new graph to find the most similar graph in the database. I have tested 100 graphs of the same category, and only 38 graphs are right. The effect is not very good. What can I do to improve it. online address:https://aistudio.baidu.com/aistudio/projectdetail/1604692

Hi - I love what you have done here. I am currently trying to use this to incorporate colorized pictures. I keep uploading colorized pictures into the dataset, but it returns a greyscale version. Where do you convert the photos into grayscale (can't find it in the code)?

Thanks for the great project. I was wondering why you set the last layer to 5 in the fully connected network? Is there any reason to do that? I couldn't find explanation in the article also.
self.fc1 = nn.Sequential( nn.Linear(8*100*100, 500), nn.ReLU(inplace=True), nn.Linear(500, 500), nn.ReLU(inplace=True), nn.Linear(500, 5))

I have not spent any time over the past couple years on this project, and I believe it is time to make it more robust, as well as improve the quality of the results.

I am looking for collaborators who would like to work on this with me.
Specifically, I am looking to:

change to larger dataset
train a larger model, with pooling layers

Adding better documentation.

Respond to this issue if you would be interested. Any help is greatly appreciated

It seems that you didn't use the variable — index in the getitem function of SiameseNetworkDataset. So are all data traversed？

I am a beginner. What I could think of is just to see the trend of loss. How did you decide when to stop training?
Does the margin need to be tuned?
I have successfully run through your codes, but the result is not so ideal as you posted...

Thanks a lot!

Hey, I tried to run the notebook in Google Colab and when training the loss behave this way:

is it normal/expected?
I did not tweak the code nor any parameters. Please advise.

Hi I am using pytorch 0.4. I use your notebook to run simulation. Thank you for your works, it really the best tutorial. After I follow your setting and just run it I have a weird result like the different people has lower dissimilarity. I try to train again with bigger epoch and check if your code input wrong pairs of image. However after I checked all is well. Your data loader is also right but I have bad result for dissimilarity where the same person has higher dissimilarity but in some cases the different person has lower dissimilarity. Should I change the siamse network architecture?, thank you.

Hello,

i am using a custom dataset which has more than 1,000 classes, and I find the formed pairs are mostly from the same class.

Is the code implemented in this repository available for such a huge dataset?