Comments (10)
@Maggab1031 yes it is correct. For example, label map alone cannot distinguish between two overlapping objects and one object.
In this image, there are many people, but without instance map it will just be one big blob of just one label.
from spade.
@ak9250 thank you!
from spade.
Oh wait, in an instance map - each object gets it's own pixel value, in a label map, each TYPE of object gets its won pixel value. Is this correct?
from spade.
I've the same question here :)
from spade.
The input label, when loaded should contain integer values like 0, 1, 2, ..., 182. Each number means the label of that pixel.
There are two ways of visualizing this. One way is treating the numbers as brightness of greyscale image. This visualization will lead to the bottom images you uploaded. But this visualization is hard to recognize because the difference among different labels is so subtle. On the other hand, you can entirely change the color of each label. For example, 0 can be painted red, 1 can be painted green, and so on. The top images you uploaded correspond this. It's customary that we use this kind of coloring of labels in papers.
We have code that can "colorize" greyscale images into colorful ones. Please use Colorize
in this code.
It's harder to convert colored label images to greyscale images, because you would have to figure out what class number each color corresponds to. Usually if you got this image from a dataset, the dataset will come with greyscale version of the label map. Is this coco-stuff?
from spade.
Ok I see. Thank you very much Taesungp that a good starting solution.
from spade.
Thanks so much for your reply, @taesungp! I now understand how these grey-scale label maps work (and yep, this is coco-stuff). But here come a few questions:
-
So we can encode labels in the pixel brightness - that would be our label map (right?). How is instance map obtained then? Do we need both to generate an image (and we don't need the ones in val_img)?
-
How can I convert colored label images to greyscale images, if I know to which object each color corresponds (like, I know that green color on my colored image corresponds to "124" greyscale brightness, which is grass in coco-stuff dataset)? Is there any Python implementations?
from spade.
I made a google colab notebook that works with testing and training coco-stuff, will add other datasets and how to train a custom dataset soon https://github.com/ak9250/SPADE-colab
from spade.
- instance map is made similar to label map. In instance map, pixel value of 0 will mean the first object, 1 the second object, and so on. Therefore, it's possible to visualize instance map just like label map, in either greyscale or colorized version. Of course, the actual values of numbers don't really mean anything. We can swap the order of objects. In fact, using
get_edges
ofmodels/pix2pix_model.py
, we compute the edge from the instance map and just use that instead of the pixel values.
The pretrained models that are trained using instance map will require instance map to generate outputs.
Lastly, in case of COCO-stuff, we created an instance map that combines the boundary of label map and original instance map. COCO dataset comes with instance map. However, the instance map does not contain boundary between different labels. For example, imagine an image of two people holding hands together on grass field under clear sky. The instance map will show the boundary between the two people, but it wouldn't show the boundary between grass field and sky. We augmented the original instance map by combining the boundaries of the instance map and label map. We plan to release this script soon.
- We do not have a python implementation to do that, because we never needed one. We were always able to start with the greyscale version. But it shouldn't be so hard to do the conversion as long as you know which color corresponds to which label number.
from spade.
Hey all, great work. I'm a little confused about the difference between an instance map and a label map. They both seem to be labeling pixels of an image. Is there a difference in purpose?
from spade.
Related Issues (20)
- How can I run this using RGB label as input ? HOT 4
- CUDA error: device-side assert triggered HOT 1
- Two update_learning_rate with same name
- RuntimeError: cublas runtime error : an access to GPU memory space failed at
- Converting to onnx HOT 2
- Choice between multi-scale discriminator and progrssive growing of GANs ?
- How many categories have you predicted for flickr data using deeplab?
- How "label_nc" in the setting works? HOT 2
- CUDA Error?? HOT 4
- Public API
- Extend to instance segmentation
- IndexError with custom dataset HOT 2
- errors with custom dataset HOT 3
- how to prepare the custom datasets?
- Issue a black background is added to the transparent image HOT 1
- GauGAN2 Paper HOT 1
- add --tf_log , program can not run HOT 1
- any pretrained models on ade20k or cityscapes with use_vae?
- Online demo app no longer on nvidia site?
- Training the model for Image to Image translation without label_nc and instance maps.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spade.