Giter VIP home page Giter VIP logo

bit-da / sepico Goto Github PK

View Code? Open in Web Editor NEW
113.0 6.0 7.0 577 KB

[TPAMI 2023 ESI Highly Cited Paper] SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation https://arxiv.org/abs/2204.08808

Home Page: https://arxiv.org/abs/2204.08808

License: Other

Python 99.77% Shell 0.23%
deeplab-resnet dense-contrastive-learning domain-adaptation semantic-segmentation transformer-network

sepico's People

Contributors

binhuixie avatar kiwixr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

sepico's Issues

training time

Since we don't have a V100, how many hours did SePiCo use deelabv2 to train GTA5→Cityscapes on the V100 to reach the best score of 61mIoU?

When release the code?

Hi, i am intresting your paper very much! When release the code?
Looking forward very much......
Thanks

About reproducing the results in Synthia to Cityscape dataset

Hi,

Thanks for your impressive work. I am interested in reproducing the results you reported in paper. I tried Synthia to Cityscape dataset and adopts DistCL based on DAFormer, but my final results is 3 mIoU lower than your results, which is 64.3mIoU. I would like to ask if the parameters you used are the same as DAFormer? Or could you please upload a copy of your parameters used?

Thanks

Something about mIoU performance

Hi,could you tell me whether the final performance of the model or the highest performance in training is reported in the paper?Such as 61.0(gtav to cityscapes).

About Performance

Hi!

Thanks for your great work! The code is easy to follow.

I got a question when I try to reproduce the result.
I try to run the experiment based on GTAV → Cityscapes with DeepLab-v2 backbone, I got 59.1 mIoU following the same configs, which is lower than 61.0 mIoU. Could you please give any clue to fix this issue?

多卡训练问题

使用多卡的时候,是不是每张显卡都会有一个ProtoEstimator

Code

When will you publish the code?
Thanks,
Shahaf

代码开放

您好!
请问对应得代码什么时候会公开?
谢谢!

Replicating the results

Hi, I would like to replicate the results represented in this paper for GTAV → Cityscapes (DAFormer based) experiment. I set the Experiment ID to 4, but I still could not get the same or close results. The results that I am getting is around 64 percent while it has been reported 70 percent in the paper. Could you please let me know if I should do any thing other than ID setting to replicate the results?

About Cityscapes-->DarkZurich

Hi,
Thank you for your wonderful work! I am more interested in the Cityscapes-->DarkZurich adaptation. However, in your run_experiments.py, there is no corresponding code implementation. Therefore, I build my own implementation of the Cityscapes-->DarkZurich code, but the results are poor, with only 28.69 MIoU on the validation set. Here is my training file, can you point out my mistakes. Thank you very much!
20230203_115623.log

about performance

Thanks for your great work, but a minor issue.
Are the results reported in your paper the average of the effects of multiple seeds? If not, is there an average for experiment 4 (gta->cityscapes and synthia->cityscapes based on SegFormer) that I can refer to? This result is preferably a fair comparison with DAFormer, that is, the test number is 1/4 of the number of times you publish code and the training resolution is 512x512. I hope to cite this result in my paper for comparison.

if Lreg < 0?

To compute proto_reg:
contrast_norm = num_classes * np.log(num_classes)
proto_sim = mean_feat.mm(mean.permute(1, 0).contiguous()) / contrast_temp
loss = torch.sum(torch.softmax(proto_sim, dim=1).log()) / contrast_norm

reg_weight * proto_reg(feat, mean, contrast_temp, contrast_norm=contrast_norm)

Why loss need < 0?
I don't understand it, Thanks!

Question about running logit of the uda

Thank you for your great work, I'm really interested in it.

Due to my poor coding ability, there are some questions about the framework.
In your config file, we can know that you define uda part as a segmentor,

class UDADecorator(BaseSegmentor):

but in the model part there is a segmentor too.
type='EncoderDecoderProjector',

Why there are two segmentors, and what's the running logit of that.
Looking forward to your reply.

UDADataset的__getitem__问题

    def __getitem__(self, idx):
        if self.rcs_enabled:
            return self.get_rare_class_sample()
        else:
            s1 = self.source[idx // len(self.target)]
            s2 = self.target[idx % len(self.target)]
            return {
                **s1, 'target_img_metas': s2['img_metas'],
                'target_img': s2['img']
            }

为什么源域和目标域的索引,一个是//一个是%?

About Synthia-Cityscapes dataset

Hi,

Thanks for your great work! I would like to know if you could provide a checkpoint file of the synthia-cityscape dataset, since I can not reproduce the results in TABLE 4 (64.3) in the paper myself.

Thanks!

Q: Effect of multi-level features.

Hi,
thanks for your great work.

Table 9 states, that applying the contrastive loss to the deepest layer (4th layer) brings the largest improvement. However, applying it to layer 1 usually helped more than applying it to layer 3. That is a very interesting observation.

Further, you presented a t-SNE plot for the target domain, and I guess that the different colors refer to different classes. Did you also compute a t-SNE plot to examine whether the domains are sufficiently mixed? (I can't find a supplementary to the paper.)

Perhaps a t-SNE analysis could guide us to decide where to apply the contrastive loss? Perhaps it also depends on the progress of the training?

Best,
Marc

Installation error

dear author:
如何安装mmcv包,用pip显示‘error: metadata-generation-failed’

Why 640x640 for training and 1280x640 for testing?

Hi, thanks for your great work of using contrastive learning to bridge the domain shift between source and target domains!

However, I am a littble bit confused of the scale of input images when training and testing.
In DAFormer, the input scale of training image is 512x512, and 1024x512 for testing, but in your setting, the scales are 640x640 and 1280x640 respectively.
Did you train the model with 512x512 and evaluate with 1024x512?
As ablated in SegFormer, training with 640x640 will be about 0.5 mIoU better than 512x512.

By the way, which one is the common practice in UDA? 640x640 or 512x512?

Looking forward to your reply.

代码开放

请问老师,代码什么时候可以公开?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.