Giter VIP home page Giter VIP logo

Comments (4)

wasd96040501 avatar wasd96040501 commented on July 22, 2024 3

@ak9250
This is a bug in ade20k_dataset.py.
Although this bug is resolved, it will come up a new bug during running ade20k model.

Have a look at how to resolve this bug.

In ade20k_datset.py, first it recursively iterate all image file, and split them into two list: image and label.

SPADE/data/ade20k_dataset.py

Lines 39 to 45 in c3e6dad

for p in all_images:
if '_%s_' % phase not in p:
continue
if p.endswith('.jpg'):
image_paths.append(p)
elif p.endswith('.png'):
label_paths.append(p)

But have a look at ade20k dataset directory structure:

➜  abbey ls
ADE_val_00000001_atr.txt      ADE_val_00000002_parts_1.png
ADE_val_00000001.jpg          ADE_val_00000002_seg.png
ADE_val_00000001_parts_1.png  ADE_val_00001001_atr.txt
ADE_val_00000001_seg.png      ADE_val_00001001.jpg
ADE_val_00000002_atr.txt      ADE_val_00001001_parts_1.png
ADE_val_00000002.jpg          ADE_val_00001001_seg.png

For every .jpg image, it will be two or higher corresponding label image (eg. ADE_val_00000001_seg.png and ADE_val_00000001_parts_1.png).
So the size of image list and label list is not identical.

Just modify these code with:

        for p in all_images:
            if '_%s_' % phase not in p:
                continue
            if p.endswith('.jpg'):
                image_paths.append(p)
            elif p.endswith('_seg.png'):
                label_paths.append(p)

will fix that.

There are another bug associated with this

SPADE/data/pix2pix_dataset.py

Lines 63 to 66 in c3e6dad

def paths_match(self, path1, path2):
filename1_without_ext = os.path.splitext(os.path.basename(path1))[0]
filename2_without_ext = os.path.splitext(os.path.basename(path2))[0]
return filename1_without_ext == filename2_without_ext

This code examine whether image and label are paired.
Because (eg.) ADE_val_00000001_seg != ADE_val_00000001,
we should add another function paths_match under Class ade20k_dataset.py

    def paths_match(self, path1, path2):
        filename1_without_ext = os.path.splitext(os.path.basename(path1))[0]
        filename2_without_ext = os.path.splitext(os.path.basename(path2))[0]
        filename1_without_ext = filename1_without_ext.replace('_seg', '')
        filename2_without_ext = filename2_without_ext.replace('_seg', '')
        return  filename1_without_ext == filename2_without_ext

After this, I think bug associated with ade20k datset will be resolved.
But you will come up with another bug with cudnn.

...
...
/opt/conda/conda-bld/pytorch_1544174967633/work/aten/src/THC/THCTensorScatterGather.cu:176: void THCudaTensor_scatterFillKernel(TensorInfo<Real, IndexType>, TensorInfo<long, IndexType>, Real, int, IndexType) [with IndexType = unsigned int, Real = float, Dims = -1]: block: [292,0,0], thread: [95,0,0] Assertion `indexValue >= 0 && indexValue < tensor.sizes[dim]` failed.
Traceback (most recent call last):
  File "test.py", line 36, in <module>
    generated = model(data, mode='inference')
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lyz/workdir/SPADE/models/pix2pix_model.py", line 58, in forward
    fake_image, _ = self.generate_fake(input_semantics, real_image)
  File "/home/lyz/workdir/SPADE/models/pix2pix_model.py", line 197, in generate_fake
    fake_image = self.netG(input_semantics, z=z)
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lyz/workdir/SPADE/models/networks/generator.py", line 89, in forward
    x = self.fc(x)
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 320, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

Then we can locate the error in:

SPADE/models/pix2pix_model.py

Line 124 in c3e6dad

input_semantics = input_label.scatter_(1, label_map, 1.0)

This bug comes up when the range of mask value is negative or greater than the input dim:

Assertion `indexValue >= 0 && indexValue < tensor.sizes[dim]` failed.

Then we need to modify this file:

SPADE/data/ade20k_dataset.py

Lines 53 to 56 in c3e6dad

def postprocess(self, input_dict):
label = input_dict['label']
label = label - 1
label[label == -1] = self.opt.label_nc

to assure segmentation mask value is in range(0, label_nc)

    def postprocess(self, input_dict):
        label = input_dict['label']
        label = label - 1
        label[label == -1] = self.opt.label_nc
        input_dict['label'] = label.clamp(0, self.opt.label_nc)

After fix that, it will not raise runtime error, But the result synthesized image looks weird.

from spade.

taesungp avatar taesungp commented on July 22, 2024 3

For label map of ADE20k, we used the scene parsing dataset in this link: http://sceneparsing.csail.mit.edu/. Direct dataset link: http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip. Put the images files ADEChallengeData2016/images/validation/*.jpg and label files ADEChallengeData2016/annotations/validatoin/*.png in the same directory, and use the command below

python test.py --name ade20k_pretrained --dataset_mode ade20k --dataroot [path_to_dataset_dir]

In the result directory, the first two outputs should look like this:

image

from spade.

hhsinping avatar hhsinping commented on July 22, 2024

I will try the dataset here:
http://sceneparsing.csail.mit.edu/

from spade.

banyet1 avatar banyet1 commented on July 22, 2024

The synthesized image from coco datasets also looks weird.

from spade.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.