Coco staff dataset and inst maps about spade HOT 7 CLOSED

I wouldn't worry too much about 0-indexing and 1-indexing stuff. It's just for easier coding across many datasets. I think some datasets were 1-indexed, but eventually to convert it to 1-hot vector, it should be changed to 0-indexed.

What do you mean by the panoptic task of coco dataset directly? I didn't understand what you mean.

In my opinion, there are mainly three tasks in image segmentation just as I said in my blog: https://blog.csdn.net/Scarlett_Guan/article/details/89918328

semantic segmentation: the simplest， classify for every pixel (foreground objects and background objects)in the picture, while if there are three persons in the picture, it can’t distinguish them.
instance segmentation: only foreground objects. If there are three persons in the picture, it can distinguish them.
panoptic segmentation: the combination and upgrades of semantic segmentation and instance segmentation. It classify for every pixel (foreground objects and background objects) in the picture. If there are three persons in the picture, it can distinguish them.
I think finally you use panoptic label in fact. And the Coco-staff dataset has panoptic annotation. So maybe you can only use the panoptic label directly?

from spade.

1. The COCO dataset was originally created with 92 labels, which only contained foreground objects. At this point, there was no label for the background. Later on, Caesar et al., augmented the dataset by adding more classes to the dataset and also labeling the background. This new dataset is called COCO-stuff. We used this new dataset, which is different from COCO dataset.
2. Instance map is not necessary, but it may help generating better images. There is instance map only for the foreground object, and it's only meaningful when two objects of the same class overlap. So it is true that in most cases, the boundary between different labels provide enough signal.
3. It's just to account for the difference between 0-index and 1-index.

from spade.

Thank you so much!

1. I misunderstood about the Coco-stuff dataset before.
   (When I read the code, the annotation only contains the instances_train2017.json, which is a annotation for instance segmentation and only contains 80 categories of thing. )
   This is because I read the script: coco_generate_instance_map.py, the annotation only contains the instances_train2017.json.
   Now I know the script is only used for generating instance map, which only for the foreground object, so it only contains 80 categories of thing.

2. Your explanation is very clear. the script coco_generate_instance_map.py is only used to draw the boundary of two objects of the same class overlap, only for the foreground object.

3. I’m still confused that I only saw 0-index in the Coco official website and other websites, while I haven’t seen 1-index before.

from spade.

Besides, since you use the 182 categories containing both foreground objects and background objects, you also use the script coco_generate_instance_map.py to generate instance map for the two objects of the same classes overlap, for the foreground objects.
Then why don’t you panoptic task of coco dataset directly? I think this task can contains all the functions above.

from spade.

I wouldn't worry too much about 0-indexing and 1-indexing stuff. It's just for easier coding across many datasets. I think some datasets were 1-indexed, but eventually to convert it to 1-hot vector, it should be changed to 0-indexed.

What do you mean by the panoptic task of coco dataset directly? I didn't understand what you mean.

from spade.

@taesungp you read the Coco dataset carefully on the official website, you will find that:

1. instance segmentation
   

2. semantic segmentation
   

3. panoptic segmentation
   

When you see the second picture and the third picture carefully. You will find that the third picture can distinguish the objects of the same class? And maybe the third picture is just what you want?

from spade.

@taesungp
Why can instance map help generating better images? In comparison to the label map in the script, the "isnt map" didn’t offer any additional information.
the "label map" is like this:

the output "isnt map" just draw the boundary of the persons and padding them. While the boundary of the persons exits originally in the label map.

So I’m so confused that I guess: maybe your comprehension about the label map is not accurate. Maybe in your opinion the "label map" is like this:

And the "isnt map" can draw the boundary of the persons.

But we can get semantic segmentation from instance segmentation, while We can’t get instance segmentation from the semantic segmentation, because we don’t have enough information.

If my comprehension is not accurate, please tell me with no hesitation!

from spade.