motokimura / spacenet_building_detection Goto Github PK

View Code? Open in Web Editor NEW

176.0 8.0 54.0 86.14 MB

Project to train/test convolutional neural networks to extract buildings from SpaceNet satellite imageries.

License: MIT License

Python 0.32% Shell 0.01% Jupyter Notebook 99.65% Dockerfile 0.02%

deep-learning chainer satellite-imagery gis spacenet-dataset semantic-segmentation

spacenet_building_detection's Issues

Error cupy.cuda.memory.OutOfMemoryError: out of memory to allocate

When trying to execute

python train_model.py ../../data/dataSplit ../../data/processedBuildingLabels/3band ../../data/buildingMaskImages

I receive the following error:

/opt/conda/envs/py3.6/lib/python3.6/site-packages/cupy/core/fusion.py:659: FutureWarning: cupy.core.fusion is experimental. The interface can change in the future.
  util.experimental('cupy.core.fusion')
GPU: 0
# Minibatch-size: 16
# Crop-size: 400
# epoch: 50

Exception in main training loop: out of memory to allocate 163840000 bytes (total 1673061376 bytes)
Traceback (most recent call last):
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/trainer.py", line 304, in run
    update()
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/updaters/standard_updater.py", line 149, in update
    self.update_core()
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/updaters/standard_updater.py", line 160, in update_core
    optimizer.update(loss_func, *in_arrays)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/optimizer.py", line 593, in update
    loss = lossfun(*args, **kwds)
  File "/workspace/src/models/unet.py", line 95, in __call__
    h = self.forward(x)
  File "/workspace/src/models/unet.py", line 61, in forward
    e2 = F.relu(self.bnc2(self.c2(e1)))
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/functions/activation/relu.py", line 141, in relu
    y, = ReLU().apply((x,))
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/function_node.py", line 257, in apply
    outputs = self.forward(in_data)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/function_node.py", line 364, in forward
    return self.forward_gpu(inputs)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/functions/activation/relu.py", line 39, in forward_gpu
    y = cuda.cupy.maximum(x[0], 0)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/cupy/core/fusion.py", line 701, in __call__
    return self._cupy_op(*args, **kwargs)
  File "cupy/core/elementwise.pxi", line 804, in cupy.core.core.ufunc.__call__
  File "cupy/core/elementwise.pxi", line 381, in cupy.core.core._get_out_args
  File "cupy/core/core.pyx", line 95, in cupy.core.core.ndarray.__init__
  File "cupy/cuda/memory.pyx", line 439, in cupy.cuda.memory.alloc
  File "cupy/cuda/memory.pyx", line 916, in cupy.cuda.memory.MemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 937, in cupy.cuda.memory.MemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 694, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 749, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
Will finalize trainer extensions and updater before reraising the exception.
Traceback (most recent call last):
  File "cupy/cuda/memory.pyx", line 731, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
  File "cupy/cuda/memory.pyx", line 664, in cupy.cuda.memory.SingleDeviceMemoryPool._alloc
  File "cupy/cuda/memory.pyx", line 394, in cupy.cuda.memory._malloc
  File "cupy/cuda/memory.pyx", line 395, in cupy.cuda.memory._malloc
  File "cupy/cuda/memory.pyx", line 67, in cupy.cuda.memory.Memory.__init__
  File "cupy/cuda/runtime.pyx", line 214, in cupy.cuda.runtime.malloc
  File "cupy/cuda/runtime.pyx", line 137, in cupy.cuda.runtime.check_status
cupy.cuda.runtime.CUDARuntimeError: cudaErrorMemoryAllocation: out of memory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "cupy/cuda/memory.pyx", line 737, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
  File "cupy/cuda/memory.pyx", line 664, in cupy.cuda.memory.SingleDeviceMemoryPool._alloc
  File "cupy/cuda/memory.pyx", line 394, in cupy.cuda.memory._malloc
  File "cupy/cuda/memory.pyx", line 395, in cupy.cuda.memory._malloc
  File "cupy/cuda/memory.pyx", line 67, in cupy.cuda.memory.Memory.__init__
  File "cupy/cuda/runtime.pyx", line 214, in cupy.cuda.runtime.malloc
  File "cupy/cuda/runtime.pyx", line 137, in cupy.cuda.runtime.check_status
cupy.cuda.runtime.CUDARuntimeError: cudaErrorMemoryAllocation: out of memory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "cupy/cuda/memory.pyx", line 743, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
  File "cupy/cuda/memory.pyx", line 664, in cupy.cuda.memory.SingleDeviceMemoryPool._alloc
  File "cupy/cuda/memory.pyx", line 394, in cupy.cuda.memory._malloc
  File "cupy/cuda/memory.pyx", line 395, in cupy.cuda.memory._malloc
  File "cupy/cuda/memory.pyx", line 67, in cupy.cuda.memory.Memory.__init__
  File "cupy/cuda/runtime.pyx", line 214, in cupy.cuda.runtime.malloc
  File "cupy/cuda/runtime.pyx", line 137, in cupy.cuda.runtime.check_status
cupy.cuda.runtime.CUDARuntimeError: cudaErrorMemoryAllocation: out of memory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "train_model.py", line 152, in <module>
    train_model()
  File "train_model.py", line 148, in train_model
    trainer.run()
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/trainer.py", line 318, in run
    six.reraise(*sys.exc_info())
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/trainer.py", line 304, in run
    update()
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/updaters/standard_updater.py", line 149, in update
    self.update_core()
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/training/updaters/standard_updater.py", line 160, in update_core
    optimizer.update(loss_func, *in_arrays)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/optimizer.py", line 593, in update
    loss = lossfun(*args, **kwds)
  File "/workspace/src/models/unet.py", line 95, in __call__
    h = self.forward(x)
  File "/workspace/src/models/unet.py", line 61, in forward
    e2 = F.relu(self.bnc2(self.c2(e1)))
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/functions/activation/relu.py", line 141, in relu
    y, = ReLU().apply((x,))
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/function_node.py", line 257, in apply
    outputs = self.forward(in_data)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/function_node.py", line 364, in forward
    return self.forward_gpu(inputs)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/chainer/functions/activation/relu.py", line 39, in forward_gpu
    y = cuda.cupy.maximum(x[0], 0)
  File "/opt/conda/envs/py3.6/lib/python3.6/site-packages/cupy/core/fusion.py", line 701, in __call__
    return self._cupy_op(*args, **kwargs)
  File "cupy/core/elementwise.pxi", line 804, in cupy.core.core.ufunc.__call__
  File "cupy/core/elementwise.pxi", line 381, in cupy.core.core._get_out_args
  File "cupy/core/core.pyx", line 95, in cupy.core.core.ndarray.__init__
  File "cupy/cuda/memory.pyx", line 439, in cupy.cuda.memory.alloc
  File "cupy/cuda/memory.pyx", line 916, in cupy.cuda.memory.MemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 937, in cupy.cuda.memory.MemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 694, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 749, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
cupy.cuda.memory.OutOfMemoryError: out of memory to allocate 163840000 bytes (total 1673061376 bytes)

cupy.cuda.compiler.CompileException: nvrtc: error: invalid value for --gpu-architecture (-arch)

Dockerfile build - miniconda issue

The Dockerfile's miniconda curl returns an empty repo and fails because the current line doesn't have the 'allow redirect' tag. You may want to update the conda RUN command to include the -L tag.

From:
RUN curl -o ~/miniconda.sh

To:
RUN curl -L -o ~/miniconda.sh

Solution credit found here

ModuleNotFoundError: No module named 'gdal'

Hi
I encountered this error in running the program:

ModuleNotFoundError: No module named 'gdal'

could you please help me out?

can not find the tif profiles in the container

The terminal can not find the tif files in the container directory of ../../data/processedBuildingLabels/3band"

aws dataset key changed

aws s3api get-object --bucket spacenet-dataset --key AOI_1_Rio/processedData/processedBuildingLabels.tar.gz --request-payer requester processedBuildingLabels.tar.gz

has become

aws s3api get-object --bucket spacenet-dataset --key AOIs/AOI_1_Rio/processedData/processedBuildingLabels.tar.gz --request-payer requester processedBuildingLabels.tar.gz

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.