anuragranj / back2future.pytorch Goto Github PK
View Code? Open in Web Editor NEWUnsupervised Learning of Multi-Frame Optical Flow with Occlusions
Home Page: https://avg.is.tuebingen.mpg.de/research_projects/back2future
Unsupervised Learning of Multi-Frame Optical Flow with Occlusions
Home Page: https://avg.is.tuebingen.mpg.de/research_projects/back2future
Hi,
thanks for sharing the code.
I have a generic question w.r.t the model's code.
I see that you have used three (non-shared weights) branches for 3 images in the lines:
https://github.com/anuragranj/back2future.pytorch/blob/master/back2future.py#L147-L166
In your experience, do you suggest having non-shared weights for image encoding?
I have noticed that PWCNet has shared weights for these branches.
I understand you did not train with pytorch. So, I am not sure if you have shared weights when you trained with Lua torch.
I am trying to understand the code and the design decisions made while designing this model.
In this regard, I have two doubts as below:
max_displacement
is kept constant at 4 for all the levels of feature maps.back2future.pytorch/back2future.py
Line 73 in a3b619a
In your opinion, do you think we have to have different max_displacement
for different levels of the network?
back2future.pytorch/back2future.py
Lines 178 to 179 in a3b619a
1.25 for level-5 flow
back2future.pytorch/back2future.py
Lines 196 to 197 in a3b619a
2.5 for level-4 flow
back2future.pytorch/back2future.py
Lines 213 to 214 in a3b619a
5 for level-3 flow
back2future.pytorch/back2future.py
Lines 230 to 231 in a3b619a
Could you please let me know why do we need these multiplicative factors?
I am sorry if this is a basic question to be asked. As the network is learnable, isn't it possible that the network learns this multiplicative factor automatically as well?
Thanks in advance!
After making the necessary institutions, I ran demo.py. The error I encountered is this> FileNotFoundError: [Errno 2] No such file or directory: 'pretrained / back2future_kitti.pth.tar'. When I look at it with normal access it gives an error > an error occurred while loading the archive. Is the error caused by the file's corruption?
Secondly, the output I got when I ran the line you gave as test > python3 test_back2future.py --pretrained-flow path / to / pretrained / model --kitti-dir path / to / kitti / 2015 / root
Traceback (most recent call last):
File "test_back2future.py", line 8, in
from path import Path
ImportError: cannot import name 'Path'
What solution can I apply for this?
Thank you from now. @anuragranj @JJanai
Hi,
In your Pytorch implementation, I don't understand why you use the forward upsampled flow for backward warping. should it be the backward upsampled flow? (warping from the past feature (feat5c) into the current frame should use the estimated flow ( past->curr) )
for example in your code of "back2future.py"
feat5b_warped = self.warp(feat5b, 0.625*flow6_fwd_up)
feat5c_warped = self.warp(feat5c, -0.625*flow6_fwd_up) # Q: flow6_fwd_up-> flow6_bwd_up?
It would be great if you answer my question.
Thank you,
Suhong
Thanks for your great job!
I try to binary the occlusion by using nn.sigmoid and np.round, but it doesn't seem to be the right way.
I want to know what is the right way to visualize occlusion?
Or can you send to me the files which occlusion visualization on KITTI, Sintel and Middlebury if you have them?
I shall be very grateful.:blush:
Thanks for your sharing~!
After read your paper, I noticed that, the occlusion estimation of your work is mapped to (0,1), and the occlusion estimation is used as the weight of photometric loss.
While in your paper, the occlusion estimation in Fig.3 is something like a binary map, and in the last of second paragraph of section 4.2, you said 'This clearly shows the benefit of ignoring misleading information in accordance with the occlusion estimates'. It is a little confusing to me. Is the occlusion estimation transformed into binary map? Or you just transform occlusion maps for visualizing?
Any reply will be appreciated~
Thanks again.
I found that pytorch no longer supports translation of lua script to torch models, is there a way to work around this problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.