How to process two canonical images about codef HOT 6 CLOSED

Can you show the canonical image, translated canonical image and the translated video? I cannot decide which step is wrong as the information is too limited.

from codef.

Thank you for your reply.

I follow the reconstruction step and get two canonical images like below.


And then I use controlnet with the same seed to generate images like below.


The dual video is

I think it is due to the inconsistency between two generated images

from codef.

The issue you're encountering is likely due to the translated image not being perfectly aligned with the original one. To address this, you might want to try the following solutions:

1. Consider using a model that promotes better alignment, such as a lineart or animelineart model. These models are explicitly designed to preserve the structure of the original image in the translated one and may offer improved alignment.
2. Another approach could be to focus solely on the foreground for inference. This can be achieved by setting the masks to null during the inference process.

from codef.

Thank you for your advice. I will try it.

from codef.

Hello, may I know which model of ControlNet you are referring to？ Thank you.

from codef.

openpose

from codef.