Comments (3)
And I use the initialization_finetuned.ckpt to run overfit.py
from flowmap.
For this scene, you can see the results we got on the bench scene in Fig. S3 in the supplemental. Here are a few reasons why the results you got might be bad:
- Running on only 15 frames means some points won't be as well constrained, since there are no long-range point tracks to constrain them. If possible, I would recommend running FlowMap on the full sequence instead.
- In general, FlowMap favors smooth depth maps over crisp ones, which is why borders of objects can sometimes have a tearing effect. If you want sharp, high-resolution depth maps, you'll have to fine-tune the depth map at a higher resolution for a large number of steps (say, 10k to 20k). This repo doesn't include code for that, but it should be pretty easy to set up. You would just have to upscale the depth from the CNN at the end of training to initialize a freely optimizable depth map (
nn.Parameter
with HxW shape), then fine-tune that. - Since FlowMap outputs a depth for every pixel, points that aren't well constrained will show up as noise in the final depth map. Such noisy depth maps work well for 3D Gaussian Splatting initialization, but aren't optimal for other use cases. To get sharp/high-quality depth maps, you could also train 3D Gaussian Splatting (or NeRF) using FlowMap's depth maps/poses as initialization.
@cameronosmith can probably comment on whether any further processing/cropping was applied to the point clouds in Fig. S3. Hope this helps!
from flowmap.
Thank you for your reply, I use 150frames to show the result as follow: I think it may be worse than showed in the paper
from flowmap.
Related Issues (20)
- Predicted depth range and metric depth setting HOT 1
- Inference demo
- About results HOT 1
- Is this method applicable for multi-camera calibration with inconsistent internal parameters? HOT 1
- Question about Depth Estimation in Your Paper HOT 2
- Practice of fine-tuning the initial camera poses and intrinsics in Nerfstudio HOT 2
- Dynamic Setting HOT 1
- About frames selection HOT 1
- About compute_consistency_mask function HOT 1
- Some questions about subsample.py HOT 2
- Using Gaussian Splatting that backpropagates into camera positions HOT 1
- A potential coding bug ? HOT 1
- I am now going through the pre-training part, but the error did not find the data set named acid, the specific error message is here, please tell me where I should find this data set. HOT 1
- Randomness in Results HOT 1
- Depth as only free variable HOT 2
- Q about Paper HOT 1
- Does this work for dynamic scenes? Like highway driving with lot of moving vehicles. HOT 1
- Reducing the overfit training time by avoiding repetition of the validation step HOT 1
- How to compute the ATE in the paper? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flowmap.