pathak22 / unsupervised-video Goto Github PK

I read the paper, downloaded the model and I have some questions:
(I am talking about the "motionSegmenter_fullModel.t7")

1. How do you provide data for inference? I understand that there is the 'trunk' which is the modified AlexNet and then there are different heads. I managed to feed it an image and then to feed the maskBranch and scoreBranch with the output from the trunk. I could figure out that only the maskBranch and scoreBranch are used, by following the execution flow which leads me to the next question:
2. How can I make the model use the colorBranch? And what is the flowBranch used for? It seems that the model in that file just has the scoreBranch in it.
3. How to interpret the numbers that the scoreBranch and maskBranch compute? I could see that maskBranch outputs a feature map with 3136 channels, but what should it be used for?
4. I had to modify the line with "model:float()" from load_motionmodel.lua to "model = model:float()" and did the same for cuda as well as the float and cuda functions in DeepMask.

Hi Deepak,

I had a question, can you share how the output mask is created, like if I have an input image of let's say 227x227x3 and get an output mask of 56 x 56, how should I apply the output mask on the image or will I be able to see proper segmentation in the mask itself and there is no need to apply it on the image. Are there some coordinates for this mask?

Thanks and Regards

Hi,
I just wanted to know, how was the validation accuracy profile when you were training a foreground/background segmentation model with AlexNet/CaffeNet architecture. From what accuracy did you start the training and what validation accuracy you obtained at the end. Were you getting low validation accuracy because of the noisy labels obtained because of the inaccuracy of the motion segmentation algorithm, or you observed a general trend of increasing validation accuracy?
Thanks,
Aditya Vora

I am trying to load the torch models but none of them seem to work. I keep getting these errors:

Warning: Failed to load function from bytecode: (binary): cannot load incompatible bytecodeWarning: Failed to load function from bytecode: [string ""]:1: unexpected symbol/root/torch/install/bin/luajit: /root/torch/install/share/lua/5.1/torch/File.lua:308: bad argument #1 to 'ipairs' (table expected, got nil)
stack traceback:
[C]: in function 'ipairs'
/root/torch/install/share/lua/5.1/torch/File.lua:308: in function 'readObject'
/root/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/root/torch/install/share/lua/5.1/nn/Module.lua:192: in function 'read'
/root/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject'
/root/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/root/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject'
/root/torch/install/share/lua/5.1/nn/Module.lua:192: in function 'read'
/root/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject'
/root/torch/install/share/lua/5.1/torch/File.lua:409: in function 'load'
load_motionmodel.lua:13: in main chunk
[C]: in function 'dofile'
/root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670

I am using this docker image: https://hub.docker.com/r/kaixhin/cuda-torch/ with tag: 8.0
It has Lua5.1, Torch7 and Luajit2.1beta installed and I believe that the error comes from the fact that these versions of the software read those files differently.

Can you please tell me what versions of the software you used or whether there are any additional setup details? Thank you.

Hi, the torch model cannot be read in torch with luajit 21 because of serialization error I am assuming. Please fix.

I got this error while running "load_motionmodel.lua"
load_motionmodel.lua:16: attempt to index local 'model' (a nil value)

I found that after loading "motionSegmenter_fullModel.t7" I only got 'nil' in model.
Any suggestion?

I tried to do inference on trained motion segmentation model with motionSegmenter_fullModel.t7.
However I could not find any input loader or sample inference codes.
I only found following piece of inference method code that seems not be able to run.

-- function: inference (used for full scene inference)
function DeepMask:inference()
   self:cuda()
   utils.linear2convTrunk(self.trunk,self.fSz)
   self.trunk:evaluate()
   self.trunk:forward(torch.CudaTensor(1,3,800,800))
   if self.flow then
      utils.linear2convHead(self.flowBranch)
      self.flowBranch:evaluate()
      self.flowBranch:forward(torch.CudaTensor(1,512,300,300))
      return
   end

   utils.linear2convHead(self.maskBranch.modules[1])
   self.maskBranch = self.maskBranch.modules[1]
   self.maskBranch:evaluate()
   self.maskBranch:forward(torch.CudaTensor(1,512,300,300))

   if self.color then
      utils.linear2convHead(self.colorBranch)
      self.colorBranch:evaluate()
      self.colorBranch:forward(torch.CudaTensor(1,512,300,300))
   else
      utils.linear2convHead(self.scoreBranch)
      self.scoreBranch:evaluate()
      self.scoreBranch:forward(torch.CudaTensor(1,512,300,300))
   end
end

could you provide sample codes for inference(generating segmentation mask from trained DeepMaskAlexNet) or explain how to do it.
Thanks.