A quick demo script for you using ControlNet without leaving Blender.

• AUTOMATIC1111/stable-diffusion-webui
• Mikubill/sd-webui-controlnet

For now, it's just a quick and hardcoded script.

You have to install the Mikubill/sd-webui-controlnet extension in A1111 and download the ControlNet models first, please refer to the installation instructions from Mikubill/sd-webui-controlnet.

Notes

• In new version of Mikubill/sd-webui-controlnet, you need to enable Allow other script to control this extension in settings for API access.

DEFAULT_PARAMS = {
    "prompt": "...",
    "negative_prompt": "...",
    "width": 512,
    "height": 512,
    "seed": -1,
    "subseed": -1,
    "subseed_strength": 0,
    "batch_size": 1,
    "n_iter": 1,
    "steps": 20,
    "cfg_scale": 7,
    "restore_faces": False,
    "sampler_index": "DPM++ 2M Karras",
    "controlnet_module": "depth",
    "controlnet_model": "diff_control_sd15_depth_fp16 [978ef0a1]", 
    "controlnet_guidance": 1.0,
    "enable_hr": False,
    "denoising_strength": 0.5,
    "hr_scale": 2,
    "hr_upscale": "Latent (bicubic antialiased)",
}

and don't forget to modify the image saving folder.

IMAGE_FOLDER = "F:\\PATH\\DOESNT\\EXIST\\YET"

Notes

• Make sure you have the right name for controlnet_model, hash does matter. You can get your local installed ControlNet models list from here.

• Blender addon