Dataset and Trimap about modnet HOT 1 CLOSED

Hi, thanks for your attention.

For your questions:

Q1: When there is an intermediate background such as a seat and other people’s avatars at both ends of the head and shoulders, the matting effect is poor, I can understand Is the semantic segmentation of portraits in the network inaccurate?
Yes. There are two main problems to cause inaccurate mattes. First, these cases are not included in our training set, so the portrait semantic segmentation is wrong. Second, our pre-trained model for image matting is not adapted to the domain of daily photos. The pre-trained models used in our WebCam demo may obtain better performance in your cases (your images appear to contain noise). To achieve this, in our Colab image matting demo, please change:

!gdown --id 1mcr7ALciuAsHCpLnrtG_eop5-EYhbCmz \

to

!gdown --id 1Nf1ZxeJZJL8Qx9KadcYYyEmmlKhTADxX \

Q2: If this is the problem, can you add some such images to it?
I am sorry that we do not have the labeled data of such images. One possible solution is to perform SOC adaptation by using the unlabeled data. However, SOC can not address the problems caused by inaccurate semantic segmentation.

Q3: You mentioned that the modnet can use trimap input, how do you do it? If the trimap is not very accurate, can the network adapt to the trimap to some extent?
If you want to use the trimap, you need to concatenate it with the RGB image as the input, which means the model should be re-trained. If you have trimap, I suggest you try the existing trimap-based methods, e.g., DIM.

Q4: There is also a portrait semantic segmentation model in the model you provided. How was it trained?
This model is the pre-trained MobileNetV2 from this repo. We initial our backbone by its weights for training.

from modnet.