Comments (41)
I found a way to remove the ghosting : https://www.reddit.com/r/StableDiffusion/comments/12lmlem/i_found_a_way_to_remove_the_ghosting_of/
from sd-cn-animation.
No. I'm not sure about the exact reason but one of the things might be that you have multiple versions of python installed on your system. Try the following, run:
py -m pip freeze
Check if there is h5py in the list. If there is not, that means that when you run 'pip' command it interacts with the different version of python that is set to 'py' command. If so, you should install h5py again with the following command:
py -m pip install h5py
from sd-cn-animation.
I made a quick test I don't have quality degradation anymore and there's less ghosting, awesome job !
from sd-cn-animation.
1 hour and 361 frames in, no color distortion and no background distortion. seems to be fixed
from sd-cn-animation.
like its animating I guess but reality has melted
from sd-cn-animation.
for the error you get every generation : "go to modules/api/api.py in web-ui and replace this string (it should be at 275 line)
script_args[alwayson_script.args_from:alwayson_script.args_to] = request.alwayson_scripts[alwayson_script_name]["args"]
with this:
for idx in range(0, min((alwayson_script.args_to - alwayson_script.args_from), len(request.alwayson_scripts[alwayson_script_name]["args"]))):
script_args[alwayson_script.args_from + idx] = request.alwayson_scripts[alwayson_script_name]["args"][idx]
Also in my case the image was become purple I fixed this bug by enable "Apply color correction to img2img results to match original colors." in settings>stable diffusion I don't know if that can help you, try differents resolution too.
I have the picture degraded a lot when LoRA is activated too
Hope that helps
from sd-cn-animation.
from sd-cn-animation.
Yes
from sd-cn-animation.
Also I need to restart automatic1111 every time I make a test with script.py
from sd-cn-animation.
nm that just needed another indent
from sd-cn-animation.
It's doesn't fix the error message sorry I don't know how to solve this problem do you have the last version of automatic1111 ?
from sd-cn-animation.
Oh ok
from sd-cn-animation.
I have still some problem with the script too, sometime it's become more and more darker every images
from sd-cn-animation.
I have still some problem with the script too, sometime it's become more and more darker every images
likewise, It seems like the control net is losing all the detail
from sd-cn-animation.
I hope that will be fixed soon this script is awesome when it works
from sd-cn-animation.
could we run another controlnet alongside hed?
from sd-cn-animation.
I don't know but I have tried canny instead of hed on a character without environment, here the result : https://www.reddit.com/r/StableDiffusion/comments/12hopoe/titok_dancer_convert_into_jenna_ortega_using/
from sd-cn-animation.
I don't know but I have tried canny instead of hed on a character without environment, here the result : https://www.reddit.com/r/StableDiffusion/comments/12hopoe/titok_dancer_convert_into_jenna_ortega_using/
interesting, what does using canny look like in the script?
from sd-cn-animation.
I put this settings in script.py :
"ControlNet":{
"args": [
{
"input_image": b64_hed_img,
"module": "canny",
"model": "control_canny-fp16 [e3fe7712]",
"weight": 0.4,
"resize_mode": "Just Resize",
"lowvram": False,
"processor_res": 512,
"guidance": 1,
"guessmode": False
}
from sd-cn-animation.
But I thinks environments will be maybe destroy with this settings
from sd-cn-animation.
For your pictures example above have you try "w,h = 480, 720" in script.py for the resolution ?
from sd-cn-animation.
I'm still playing around with settings, I've tried a couple resolutions but I'm exploring other parameters to see what can help
from sd-cn-animation.
I tested a older version of script.py and I don't have image quality degradation with this version : https://raw.githubusercontent.com/volotat/SD-CN-Animation/c75ea0c5e7f44a5ae4f3323f58447beccb879a91/script.py
from sd-cn-animation.
But the temporal consistency and probably not as good the last version
from sd-cn-animation.
I'll give it a try. thinking of the color problem, maybe the vae isn't staying loaded with each frame inference?
from sd-cn-animation.
I dont know, also do you have installed the requirements for RAFT maybe it's can fix the problem :
conda create --name raft
conda activate raft
conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.1 matplotlib tensorboard scipy opencv -c pytorch
from sd-cn-animation.
yes I did that RAFT setup. Also I think the old script.py didn't use RAFT either, I think thats the reason there is more temporal consistency is RAFT
from sd-cn-animation.
is it faster/better without RAFT? i know it uses less VRAM, but is there a tradeoff when using RAFT
from sd-cn-animation.
I'm having the same problem, the image degrades after the first frame. I've made sure I had all RAFT dependencies, also tried the previous version of the script linked above, but it degrades the frames as well. Very frustrating because really want to use this!
from sd-cn-animation.
Please, check out the last version of the script and tell me if the issue with quality degradation is still prevalent.
from sd-cn-animation.
I have a error when I try to compute the flow :
File "C:\AUTOMATIC1111\SD-CN-AnimationV4\compute_flow.py", line 8, in
import h5py
ModuleNotFoundError: No module named 'h5py'
I have installed hp5py with "pip install h5py" but it's doesn't work
For the command I have write : py compute_flow.py -i {C:\AUTOMATIC1111\SD-CN-AnimationV4\1.mp4} -o {C:\AUTOMATIC1111\SD-CN-AnimationV4\output *.h5} -v -W {720} -H {480}
from sd-cn-animation.
You should not put brackets at the parameters, it is just an indication that you should modify it. Try it like this
py compute_flow.py -i C:\AUTOMATIC1111\SD-CN-AnimationV4\1.mp4 -o C:\AUTOMATIC1111\SD-CN-AnimationV4\output_flow.h5 -v -W 720 -H 480
from sd-cn-animation.
I have still the "No module named 'h5py'" error maybe I need to restart my computer ?
from sd-cn-animation.
You have right it's fixed my problem now it's works
from sd-cn-animation.
running a longer test now, will take about 2 hours to complete (698 frames at 400x600)
question, is it possible that using Euler as opposed to DPM 2S a would cause the issue?
ive switched to Euler a incase its a lack of ancestral that caused it too.
from sd-cn-animation.
The degradation seems to be resolved! thank you. Just to confirm, the ghosting effect is the result of too low an FPS correct? higher framerates help remove it?
from sd-cn-animation.
Yeah, you can see it in the optical flow window when you run compute_optical.py.
The more white you see the more movement per frame.
And the more ghosting.
I wonder if interpolation would help, as SD is going to re-create the images anyway, any random mid frames created might dissappear with the img2img process.
from sd-cn-animation.
I'm still running a 30fps test, I'll try 60/120 soon
from sd-cn-animation.
do you think open pose is worth trying as well? we are only using canny by default but I was thinking about trying a multi-controlnet scheme. also the release of controlnet1.1 is exciting, I hope they bring it to a1111 quickly
from sd-cn-animation.
For the ghosting problem maybe if I use the super slow motion of Runwayml for generate a very high frame rate video it's can fix the problem, but obviously the render time will be extremely slow with a high numbers of images and the flow file too big...
from sd-cn-animation.
with the recent background removal update, are we happy to close this issue or does it need further work?
from sd-cn-animation.
Related Issues (20)
- bug: TypeError: 'NoneType' object is not iterabl HOT 7
- An exception occurred while trying to process the frame: unsupported operand type(s) for +: 'int' and 'NoneType' HOT 1
- IS THERE A PLACE TO UPLOAD STRICTLY SD-CN ANIMATION? HOT 2
- ControlNet no longer working HOT 11
- Previous Image Quality severely dropped on fully processed image
- FileNotFoundError HOT 1
- An exception occurred while trying to process the frame: HTTP Error 503: Service Unavailable HOT 1
- SD-CN-Animation base_ui.py error
- No such file or directory: C:\Users\<name>\.cache\torch\hub\checkpoints\models\FloweR\... HOT 2
- [BUG]: it's "skimage" NOT "scikit-image" !!! HOT 1
- Does not work with sdxl
- A Gallery of Sorts? HOT 1
- This error keeps popping after a few renders: 2023-10-24 09:27:13 INFO [httpx] HTTP Request: POST http://localhost:7860/reset "HTTP/1.1 200 OK" HTTP Request: POST http://localhost:7860/reset "HTTP/1.1 200 OK"
- SD-CN causing "No ControlNetUnit detected in args" ERROR HOT 4
- [BUG] Error when Generating HOT 6
- Output is all messed up
- Notice: extension is incompatible with webui 1.8, pending removel form extension index HOT 5
- No module named 'basicsr' HOT 1
- No module named 'basicsr
- txt2vid doesn't work at all HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sd-cn-animation.