Giter VIP home page Giter VIP logo

surfacenet's Introduction

News

SurfaceNet

M. Ji, J. Gall, H. Zheng, Y. Liu, and L. Fang. SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis. ICCV, 2017

The poster pdf is also available.

SurfaceNet experiment results SurfaceNet pipeline

How to run

  1. install Nvidia driver 375 + cuda 8.0 + cudnn v5.1
  2. install the conda environment by: bash installEnv.sh
    • DON'T WORRY, conda will generate an isolated environment for SurfaceNet with python2.7, anaconda, theano, ... etc. That means all your libraries / packeges' version will not be affacted, at the same time the ~/.bashrc file will not be changed.
    • before you run, PLEASE change the CUDA/CUDNN path in the files:
      • ./config/activate-cuda.sh change the 1st line to your cuda path, e.g.: export CUDA_ROOT=/usr/local/cuda
      • ./config/activate-cudnn.sh change the 1st line to your cudnn path, e.g.: export CUDNN_ROOT=/home/<your-user-name>/libs/cudnn
  3. download the network model to the folder "./inputs/SurfaceNet_models" from the Dropbox folder
  4. if the conda environment has been installed, one can activate it by: . activate SurfaceNet; deactivate it by: . deactivate.
  5. in terminal run: python main.py

Evaluation results

Some evaluation results are uploaded, including '.ply' files and the detailed number of Table 3. This could be helpful if you want to compare with this work.

License

SurfaceNet is released under the MIT License (refer to the LICENSE file for details).

Citing SurfaceNet

If you find SurfaceNet useful in your research, please consider citing:

@inproceedings{ji2017surfacenet,
  title={SurfaceNet: An End-To-End 3D Neural Network for Multiview Stereopsis},
  author={Ji, Mengqi and Gall, Juergen and Zheng, Haitian and Liu, Yebin and Fang, Lu},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
  pages={2307--2315},
  year={2017}
}

surfacenet's People

Contributors

mjiust avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

surfacenet's Issues

Some installation issues

I just installed your code and ran it succesfully. Here are some comments:

  • You should mention that your install script requires us to allow the miniconda install to append to our .bashrc - I didn't allow this and it took me a bit to figure out why the rest of the script wasn't working.
  • you are sourcing .bashrc without the ~/ in
    . .bashrc # to enable the conda command

    which just always fails
  • the else clause is commented here:
    echo "The '$conda_env_name' conda env exists." # but the env will be visible even thoug the installation was terminated. In this case you should delete this conda env first.else # if the env does not exist.

aside from those, I got everything working pretty smoothly

Problem about theano 0.9.0

I installed lasagne 0.1 and theano 0.9 at first, but it doesn't work
ImportError: Could not import Theano. Please make sure you install a recent enough version of Theano.
Then I installed lasagne 0.2.dev1 and theano 1 instead but theano.sandbox.cuda.dnn is discarded in theano 1(like #3 )
import similarityNet File "./nets/similarityNet.py", line 6, in <module> if lasagne.utils.theano.sandbox.cuda.dnn_available(): # when cuDNN available AttributeError: 'module' object has no attribute 'cuda'
@mjiUST could U help me? thanks

Cant works on theano 1,theano.sandbox.cuda.dnn is discarded in new version

Could you update your source file layer.py?
Because theano.sandbox.cuda.dnn is discarded in theano 1(>theano 0.9).
from theano.sandbox.cuda.dnn import gpu_contiguous, GpuDnnConvDesc, gpu_alloc_empty, GpuDnnConv3dGradW wont work, and if lasagne.utils.theano.sandbox.cuda.dnn_available() in similarityNet.py.
Could you use theano.gpuarray.dnn instead?
I cant replace gpu_contiguous, GpuDnnConvDesc, gpu_alloc_empty, GpuDnnConv3dGradW with classes of theano.gpuarray.dnn by myself.
And I cant backwards to theano 0.9 either, due to the new vision of cudnn does not support old theano and pygpu.
plz help me,thanks

Error while running main.py

I'm running this code in my Amazon AWS server
Configuration of my system
`==============NVSMI LOG==============

Timestamp : Tue Apr 24 17:43:40 2018
Driver Version : 375.66

Attached GPUs : 1
GPU 0000:00:1E.0
Product Name : Tesla K80
Product Brand : Tesla
CUDA versionnvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61
`

So when I run python main.py
I got this error

attached as log file
log.txt

How to generate the pos file?

Thanks for the great released code! I am wondering what the detailed information recorded in the pos file is. Is that an essential matrix or a homography matrix? And how can we generate it by ourselves? Eager for your reply.

terminated,exit value:139 (Segmentation fault (core dumped))

After I follow this #4, the enviroment problem seems to be gone.
But evertime python main.py occour Segmentation fault (core dumped), the python interpreter stops on main_reconstruc.py line 73
patches_embedding, inScope_cubes_vs_views = earlyRejection.patch2embedding(
images_list, img_h_cubesCorner, img_w_cubesCorner, patch2embedding_fn, patches_mean_bgr,
N_cubes, N_views, D_embedding, patchSize = params.__imgPatch_hw_size,
batchSize = params.__batchSize_similNet_patch2embedding,
cubeCenter_hw = np.stack([img_h_cubesCenter, img_w_cubesCenter], axis=0)) # (N_cubes, N_views, D_embedding), (N_cubes, N_views).

output like this
start reconstruction ...
loaded img ./inputs/Middlebury/dinoSparseRing/dinoSR0007.png
loaded img ./inputs/Middlebury/dinoSparseRing/dinoSR0008.png
loaded img ./inputs/Middlebury/dinoSparseRing/dinoSR0009.png
loaded img ./inputs/Middlebury/dinoSparseRing/dinoSR0010.png
loaded img ./inputs/Middlebury/dinoSparseRing/dinoSR0011.png
loaded img ./inputs/Middlebury/dinoSparseRing/dinoSR0012.png
xyz bounding box of the reconstructed scene: [-0.061897 0.010897], [-0.018874 0.068227], [-0.057845 0.015495]
loaded similarityNet model: ./inputs/SurfaceNet_models/epoch33_acc_tr0.707_val0.791.model
loaded SurfaceNet model: ./inputs/SurfaceNet_models/2D_2_3D-19-0.918_0.951.model
Segmentation fault (core dumped)

It seems like a stack or memery problem. Can you help me? Any advice?
My OS is
Ubuntu 1604 x64
cuda 8.0+ cudnn5.1 and cuda 9.0+cudnn7.1.1
1080Ti

.theanorc
[global]
floatX=float32
device=cuda
optimizer=None

allow_gc=True
gpuarray.preallocate=0.75
gcc.cxxflags=-Wno-narrowing
exception_verbosity=high

[lib]
cnmem=0.75

[nvcc]
nvcc.fastmath=True

[cuda]
root=/usr/local/cuda-8.0

@mjiUST @Rubikplayer

CUDNN_STATUS_INTERNAL_ERROR while running main.py

Hi thanks for previous feedback in another thread. After I setup up Cuda8.0/CuDNN 5.1 and theano 0.9, I can run some part of main.py. But there's still some error when executing patch2embedding() function in the early rejection stage.

More specifically:

Traceback (most recent call last):
  File "./main.py", line 27, in <module>
    save_npz_file_path = main_reconstruct.reconstruction(datasetFolder, _model, imgNamePattern, poseNamePattern, outputFolder, N_viewPairs4inference, resol, BB, viewList)
  File "/home/ICT2000/tli/Workspace/SurfaceNet/main_reconstruct.py", line 77, in reconstruction
    cubeCenter_hw = np.stack([img_h_cubesCenter, img_w_cubesCenter], axis=0))    # (N_cubes, N_views, D_embedding), (N_cubes, N_views)
  File "./utils/earlyRejection.py", line 31, in patch2embedding
    patches_embedding[:,:] = patch2embedding_fn(patch_allBlack)[0] # don't use np.repeat (out of memory)
  File "/home/ICT2000/tli/.conda/envs/SurfaceNet/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/ICT2000/tli/.conda/envs/SurfaceNet/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/ICT2000/tli/.conda/envs/SurfaceNet/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
RuntimeError: error doing operation: CUDNN_STATUS_INTERNAL_ERROR
Apply node that caused the error: GpuDnnConv{algo='small', inplace=False}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty{dtype='float32', context_name=None}.0, GpuDnnConvDesc{border_mode=(1, 1), subsample=(1, 1), conv_mode='cross', precision='float32'}.0, Cast{float32}.0, Cast{float32}.0)
Toposort index: 276
Inputs types: [GpuArrayType<None>(float32, (False, False, False, False)), GpuArrayType<None>(float32, (False, False, False, False)), GpuArrayType<None>(float32, (False, False, False, False)), <theano.gof.type.CDataType object at 0x7fbd6848bc90>, Scalar(float32), Scalar(float32)]
Inputs shapes: [(1, 3, 64, 64), (64, 3, 3, 3), (1, 64, 64, 64), 'No shapes', (), ()]
Inputs strides: [(49152, 16384, 256, 4), (108, 36, 12, 4), (1048576, 16384, 256, 4), 'No strides', (), ()]
Inputs values: ['not shown', 'not shown', 'not shown', <capsule object NULL at 0x7fbb43bd10c0>, 1.0, 0.0]
Inputs type_num: [11, 11, 11, '', 11, 11]
Outputs clients: [[HostFromGpu(gpuarray)(GpuDnnConv{algo='small', inplace=False}.0)]]

Detail error log can be seen here:
err_log.txt

I have tried:

None has worked so far.

Have you seen this type of error before? Or did I set my computer correctly?
I observed you have a params.py to specify all parameters. Some has mentioned this error can result from lack of memory (link), and it seems your code did something for batch processing.

Info of my setting:

  • Ubuntu 16.04
  • CUDA 8.0 / CuDNN 5.1
  • GPU: Nvidia 1080 Ti (11GB memory) --- I also tried on another machine with Titan X, not working
  • theano 0.9

My ~/.theanorc:

[global] 
floatX=float32 
device=cuda0
optimizer=None
 
allow_gc=True 
#gpuarray.preallocate=0.95
gcc.cxxflags=-Wno-narrowing
exception_verbosity=high

[lib]
cnmem=0.75

[nvcc]
nvcc.fastmath=True 

[cuda] 
root=/usr/local/cuda-8.0

If you have any suggestions, please let me know! Thanks for your help and support!


Update:

After I tried to remove other versions of CuDNN: (https://groups.google.com/forum/#!topic/theano-users/w4M3Xy0ec60), the error changes to the following.

Traceback (most recent call last):
  File "./main.py", line 27, in <module>
    save_npz_file_path = main_reconstruct.reconstruction(datasetFolder, _model, imgNamePattern, poseNamePattern, outputFolder, N_viewPairs4inference, resol, BB, viewList)
  File "/home/ICT2000/tli/Workspace/SurfaceNet/main_reconstruct.py", line 77, in reconstruction
    cubeCenter_hw = np.stack([img_h_cubesCenter, img_w_cubesCenter], axis=0))    # (N_cubes, N_views, D_embedding), (N_cubes, N_views)
  File "./utils/earlyRejection.py", line 48, in patch2embedding
    _patches_embedding_inScope[_batch] = patch2embedding_fn(_patches_preprocessed[_batch])     # (N_batch, 3/1, patchSize, patchSize) --> (N_batch, D_embedding). similarityNet: patch --> embedding
  File "/home/ICT2000/tli/.conda/envs/SurfaceNet/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/ICT2000/tli/.conda/envs/SurfaceNet/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/ICT2000/tli/.conda/envs/SurfaceNet/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
  File "pygpu/gpuarray.pyx", line 676, in pygpu.gpuarray.pygpu_empty
  File "pygpu/gpuarray.pyx", line 290, in pygpu.gpuarray.array_empty
pygpu.gpuarray.GpuArrayException: cuMemAlloc: CUDA_ERROR_OUT_OF_MEMORY: out of memory
Apply node that caused the error: GpuDnnConv{algo='small', inplace=False}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty{dtype='float32', context_name=None}.0, GpuDnnConvDesc{border_mode=(1, 1), subsample=(1, 1), conv_mode='cross', precision='float32'}.0, Cast{float32}.0, Cast{float32}.0)
Toposort index: 276
Inputs types: [GpuArrayType<None>(float32, (False, False, False, False)), GpuArrayType<None>(float32, (False, False, False, False)), GpuArrayType<None>(float32, (False, False, False, False)), <theano.gof.type.CDataType object at 0x7f703a019c90>, Scalar(float32), Scalar(float32)]
Inputs shapes: [(1100, 3, 64, 64), (64, 3, 3, 3), (1100, 64, 64, 64), 'No shapes', (), ()]
Inputs strides: [(49152, 16384, 256, 4), (108, 36, 12, 4), (1048576, 16384, 256, 4), 'No strides', (), ()]
Inputs values: ['not shown', 'not shown', 'not shown', <capsule object NULL at 0x7f6e133930c0>, 1.0, 0.0]
Inputs type_num: [11, 11, 11, '', 11, 11]
Outputs clients: [[HostFromGpu(gpuarray)(GpuDnnConv{algo='small', inplace=False}.0)]]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.