Comments (6)
I think it will pose some problems because with warping you will compare image parts with black areas. The optimizer will try to avoid it to minimize photometric error.
To my mind, your best solution is to weight the photometric error for each pixel. If you know exactly the pixels to dismiss, just multiply these photometric loss values by 0
You can find an example of how I did the same thing with out of bound warped pixels in loss_functions.py
from sfmlearner-pytorch.
Thank you for your answer! :)
That makes a lot of sense. I will implement it and get back to you as soon as possible!
from sfmlearner-pytorch.
Hi, @ClementPinard
I have carefully read how you did for the out of bound warped pixels in loss_functions.py
, but I am wondering when the pose or depth is not very accurate, such as at the begin of training, there might be a lot of pixels out of bound. Since we multiply these photometric loss values by 0, the photometric loss will be quite small. I think it will lead the network to divergence.
I have this problem when I use it in other scenarios. Do you have any experience dealing with it?
from sfmlearner-pytorch.
Normally it should not be a problem. At first, pose values are very low, so warping is very close to the identity function, which makes the gradient meaningful.
If you actually have out of bound pixels, then the difference won't have any gradient, because you are comparing a valid pixel to a grey zone on which no gradient is possible.
So discarding it from the loss value is just a way to have a more meaningful loss since you only deal with valid comparisons. This makes sense at the end of the training, where half of warping is zooming out to warp, with obvious out of bound pixels in the boundaries of the target image.
from sfmlearner-pytorch.
What scenario are you using it on ? A known problem is when translation is not enough, then the depth has very little influence on the warping since the parallax is low.
You can try the train_flexible_shift.py
script which tries to overcome this problem by increasing temporal shift between frames when the translation predicted is too low.
from sfmlearner-pytorch.
Thank you for your prompt reply. I think you are right and I have fixed my problem. The divergence in my work is just because of a stupid typo.
from sfmlearner-pytorch.
Related Issues (20)
- what's the minimal files required to train depth only model HOT 1
- Query regarding depth map. HOT 2
- Large Errors on Pose Prediction Network HOT 3
- why the gpu memory cost of tensorflow version is larger than pytorch version HOT 2
- Weird results from pretrained model on KITTI images HOT 4
- Question about using oxts data HOT 1
- Cannot run `train.py` with nohup HOT 2
- imread during inference load the image as uint8 HOT 4
- How About the Flops, fps and parameter of this model? HOT 1
- regarding the predicted depth map during inverse warp HOT 2
- How to visualize the warped image (ref_img_wapred) HOT 2
- regarding inverse_warping HOT 10
- Is the image input of depth network fixed? HOT 2
- Question about diff
- How to load training dataset
- Regarding the depth used for generating target image HOT 5
- Question about the poses predicted by the posenet HOT 2
- about the pose scale HOT 2
- difference of the predicted translation and ground truth vectors HOT 7
- samples in test_files_eigen dont exist in the KITTI
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sfmlearner-pytorch.