porting to the newest caffe about hed HOT 26 OPEN

Hi all, I've reimplemented hed based on new caffe, check out it here https://github.com/zeakey/hed!

from hed.

Hi @s9xie @saedrna @fxw6000 @BingTSC @juntingzh @stevendbrown

Actually the shift at predicted edge map are due to the different implementation of CropLayer between HED and official branch of caffe.

In HED version of caffe which I think is based on Longjon's FCN, the crop offset is computed automatically inside the network, that is, in crop layer you will not need to specify the offset. But this implementation is very complicated and has a lot of modifications on almost all layers cpp code.

In the newly official branch of caffe, the crop layer has a parameter named "offset", this allow you to specify the crop offset manually, sounds not good because you have to compute offset by yourself, but caffe now provide NetSpec which you can generate net prototxt file by a python script, in NetSpec it will compute the crop layer offset automatically, I think this is a good solution which avoid modifying the C++ layer implementations and provide automatically offset computation.

Berkeley.FCN is a example of NetSpec script that generate net prototxt file which contain a crop layer, and here is my reimplementation of HED network in NetSpec, it is totally compatitable with the newest version of caffe.

from hed.

my hack port of the ImageLabelmapData to the current version of caffe is here:

https://github.com/stevendbrown/caffe/tree/HED

i set the padding in train_val.prototxt to 1 and get excellent results with de novo training and prediction, albeit on a relatively simple dataset. my non-quantified impression is that with the current caffe branch i can increase the training batch size without the performance hit i noticed if i increased the batch size using the HED version of caffe.

i'm not a computer scientist, so if anyone can comment on the chop-shop job i've done here, i'd be grateful.

from hed.

Hi all,

First I want to really apologize for the slow response in the repo issues.

@saedrna @zeakey @fxw6000 First that's right you need to have the padding in the first layer to avoid the shifting problem. And it is the correct behavior that you are detecting the image borders as boundaries.
To solve this problem, what I was doing is a simple trick:

During testing stage, I "augment" the testing image by padding it with mirror reflections (both vertical and horizontal) of itself. Then after you get the edge map, you can just crop the center image. I'll put a note in Readme.

I'm currently a little bit swamped with some other projects but I'll keep this open and let you know when I migrate this to the new caffe.

from hed.

@stevendbrown , the porting is simple, because recent caffe support the all the layers in testing HED, except for a simple bug (the effects and solution are described above). So I will only enlarge the image outside caffe using copyMakeBorder and after hed, subseting the corresponding region.

And caffe is also ported to windows by in a separate branch.

from hed.

May the different implementation in crop layer cause this issue.

from hed.

I have the same problem.

from hed.

Actually I don't know why a customized Caffe is used? Even for training, any special layers are designed and used? No clue is found by looking at the prototxt only. Anyone has any idea?

from hed.

@s9xie Thanks, I think for the time being, I can also use hed by the same trick. That is set pad the image manually with mirror reflection (using opencv) and then set pad=1 for the first layer rather than 35.

Looking forward to your porting to the caffe master repo!

from hed.

@saedrna, can you make your port available?

from hed.

@saedrna if you can help me get on the right track i'd be grateful. i started by attempting to apply the changes in caffe PR #4159 but there seem to be some asynchronies between the caffe branch that PR applied to and the caffe source included with hed which i haven't resolved yet. my alternative strategy was going to be to port the changes to caffe included with hed into the current caffe master branch.

maybe i'm misunderstanding you - do you mean that the current caffe master branch is a drop-in replacement for the caffe version included with hed (with the exception of the border issue described above)?

from hed.

@stevendbrown , that's right, I just used the model, prototext and changed the pad to 1, the results are reasonable, except for the boarder problem.

On windows, the windows branch of caffe is used.

from hed.

@saedrna i'm replicating your result with the padding. thanks for the pointer!

from hed.

it looks like the way layers are defined has changed with the current master branch of caffe. have any of you ported ImageLabelmapData to the current caffe?

i've moved ImageLabelmapData into the current caffe master branch and it seems to be working. if there is no code available for other people's ports, i can make a fork/PR/something with the ported code.

from hed.

@saedrna I build the caffe on windows, then how to run HED edge detection on images? I don't know what kind of modification should apply to top of Caffe Windows branch. Could you please help me? thanks.

from hed.

@BingTSC ,just use the deploy prototxt, and change pad = 35 to 1, then you will reproduce the results as above.

from hed.

adding crop_params { axis: 2 offset: N offset: N } (where N = padding) to the Crop layers also seems to work well.

from hed.

@stevendbrown @s9xie The new ported caffe works and the build is successful 👍. But the edges are not as clear as they should be like the original HED caffe. Is there anyway to make them more accurate?

from hed.

@stevendbrown hai ,I clone your git code,but when I run ./solver , I get Nan . it's ok when I run s9xie's
do you konw why .
I set the pad = 1 . and the offset 1 in cov2 , 2 in cov3 , 4 in cov4 , 8 in cov5

from hed.

@stevendbrown Hi I used your caffe to train hed. However, the output looks all dark. I changed pad from 35 to 1 in train_val and deploy. Did I miss anything? Can you share your train_val.prototxt?

from hed.

@zeakey
thanks for your contribution,then after using hed-caffe-netspec.py to produce three .pt file,what should i do next step?

from hed.

@zeakey
can u share your trainval.prototxt? after i modify my trainval.prototxt accoring to hed-train.pt
the loss is zero
thanks!

from hed.

@zeakey
according to hed-train.pt
data layer type is Input, but where should i input the data route？
hope to get your respond

from hed.

@zeakey
thanks for your code hed-caffe-netspec.py,
after i use hed-caffe-netspec.py to produce hed-train.pt, and modify my trainval.prototxt accoring to hed-train.pt, throung i set new_width=500 and new_height=500 in ImageData layer, it seems there still exist problem in crop layer,
can u share your tranval.prototxt and solve.py? i will be grateful
thx!!!!!!

from hed.

@hustzxl
hi
i meet the same problem
can u share your method to solve this problem?
thx!!!!!!!

from hed.

@zeakey
thanks a lot!!!!!

from hed.