self.image_cross_attention_scale==scale_scalar？ about stylecrafter HOT 3 CLOSED

I got it. Thank you! Yours, Miaomiao Dai 发件人: Liu Gongye 发送时间: 2023年12月20日 12:14 收件人: GongyeLiu/StyleCrafter 抄送: dmmSJTU; Author 主题: Re: [GongyeLiu/StyleCrafter] self.image_cross_attention_scale==scale_scalar？ (Issue #2) Hi, thank you for your interest. This line seems to come from the Class CrossAttention. While in StyleAdapter, we use Class DualCrossAttnProcessorAS instead of CrossAttention. So the actual fusion operation is: https://github.com/GongyeLiu/StyleCrafter/blob/d31b8be5ff3596f2323473c5f35f0c5f1888370a/lvdm/modules/attention.py#L846 And the full code: if scale_scalar is not None: scale = 1 + scale_scalar[:, self.layer_idx] scale = scale[:, None] else: scale = 1.0 if self.use_norm: out_style = self.norm_style(out_style) out = out + scale * out_style * self.scale Here, the scale is obtained from the scale_scalar(model.adapter.scale_predictor), and self.scale is provided to allow further control of the style strength. Since in most cases, users do not need to manually adjust it, we set self.scale to 1 by default, which means it completely relies on the scale_scalar to control the fusion operation. (By the way, we have never used self.scale in our paper.) Hope this answers your question. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>