mattpoggi / depthstillation Goto Github PK

Good evening,
I'd like to reproduce your depthstillation process, but on real dataset called CADDY in order to evaluate some models (pwcnet and raft).
I already extracted the depth maps through MIDAS and depthstilled them, but it seems that the metrics of the models that i trained on these depthstilled data are worsening a lot...
I'm attaching, from top to bottom, computed flow, depth map, original and depthstilled images.

Do you see something strange in them? Any tips about the use case?
Thanks in advance!

Thanks for your great work! But I have a small question about the moving object in the paper. Table 1 shows that adding moving objects gives better performance. However, in Table 2 and the following tables, the dCOCO does not appear to contain moving objects (with EPE=3.81 on KITTI 15). Do you implement the following experiments, including dDAVIS and dKITTI, without moving objects? This is very important to us because our recent work uses depthstillation as an important reference. We would appreciate receiving your reply.

I use MiDaS and follow the instructions in https://pytorch.org/hub/intelisl_midas_v2/ to get the depth image, white the depth of the demo image I get is as follows:

It seems very different from the depth image provided in this repo:

the code I use is like this:

midas = torch.hub.load("intel-isl/MiDaS", "MiDaS").cuda()
midas_transforms = torch.hub.load("intel-isl/MiDaS", "transforms")
transform = midas_transforms.default_transform

image = np.asarray(Image.open(input_image))
img = transform(image)
with torch.no_grad():
    prediction = midas(img.cuda())
    # prediction = torch.nn.functional.interpolate(
    #     prediction.unsqueeze(1),
    #     size=image.shape[:2],
    #     mode="bicubic",
    #     align_corners=False,
    # ).squeeze()
depth_image = prediction.cpu().numpy().squeeze()
depth_image = cv2.convertScaleAbs(depth_image, alpha=255/np.max(depth_image))
Image.fromarray(img, mode="L").save("demo.png")

Could you please give some details about how to get the depth image?

Hello,
Thank you for your great idea and code.
How can we apply your code to a custom dataset? I have video frames and want to generate the optic flow for my dataset. How can I use your code? Thank you.

Hi! Thanks for your great work.

I try to run the code in win10, and use gcc -shared -o libwarping.dll warping.c instead of compile.sh. I modified lib = cdll.LoadLibrary("external/forward_warping/libwarping.so") to lib = cdll.LoadLibrary("external/forward_warping/libwarping.dll"). Then I run depthstillation.py but obtain strange result like that (dCOCO/im1):

which is quite different from the result in supply. material. Did I miss something?

I think it should be Random angles in -pi/18,pi/18, excluding -pi/36,pi/36 to avoid zeros / very small rotations.

Hello,
Thank you for your great code.
How did you train the models for optical flow where the ground truth should be in .flow but your code generates .png format?

Thank you.

With command python depthstillation.py, I have tried to get warp of your sample image im0 and the first cat image in your article with code, but the result was rather strange. The cat image was a screenshot from your article and its depth map was gotten using Miads with weight "dpt_large". Here is link to the warp result of this two image im1_warped and cat_warped. Am I doing somthing wrong?

Hi, thanks for your very interesting work. Can you provide the dataset you developed?