Giter VIP home page Giter VIP logo

comfybox's Introduction

ComfyBox

ComfyBox is a frontend to Stable Diffusion that lets you create custom image generation interfaces without any code. It uses ComfyUI under the hood for maximum power and extensibility.

Screenshot

Screenshot

Installation

  1. Download the latest release here and extract it somewhere.
  2. Start the ComfyUI backend with python main.py --enable-cors-header.
  3. In the folder you extracted open the run.bat/run.sh script (requires Python 3 to be on your PATH). Alternatively you can serve the contents of the folder with any web server.

Usage

A preconfigured workflow is included for the most common txt2img and img2img use cases, so all it takes to start generating is clicking Load Default to load the default workflow and then Queue Prompt.

You can import your existing workflows from ComfyUI into ComfyBox by clicking Load and choosing the .json or .png with embedded metadata, or dropping either file onto the graph viewer.

NOTE

This project is still under construction and some features are missing, be aware of the tradeoffs if you're interested in using it.

Also note that the saved workflow format is subject to change until it's been finalized after enough testing, so be prepared to lose some of your work from time to time.

Features

  • No-Code UI Builder - A novel system for creating your own Stable Diffusion user interfaces from the basic components.
  • Manage Multiple Workflows - You can open as many workflows as you like and switch between them using tabs within the app. You can also write custom "Send To" actions to copy your image outputs into other workflows for further processing.
  • Use Your Existing Workflows - Import workflows you've created in ComfyUI into ComfyBox and a new UI will be created for you.
  • Extension Support - All custom ComfyUI nodes are supported out of the box.
  • Prompt Queue - Queue up multiple prompts without waiting for them to finish first. Inspect currently queued and executed prompts.
  • Prompt History - Browse through previously generated prompts and their output images/parameters.
  • Mobile-Friendly Version - Includes a version of the UI optimized for mobile use, while still supporting the same customized workflows of the desktop version.

Development

Requirements

  • git
  • pnpm
  • An installation of vanilla ComfyUI for the backend

Installation

NOTE: If you're using Windows, the following commands must be run with Git Bash.

  1. Clone the repo with submodules:
git clone https://github.com/space-nuko/ComfyBox --recursive
  1. pnpm install
  2. pnpm prebuild
  3. pnpm dev
  4. Start ComfyUI with python main.py --enable-cors-header
  5. Visit http://localhost:3000 in your browser

comfybox's People

Contributors

haze avatar space-nuko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

comfybox's Issues

Bookmark prompts/workflows

For example if I want to test new models in a standardized way, I can load the exact settings from a sidebar and insert the new model in without hunting for it a lot

Widget variations

Instead of having UI.Slider and UI.Number nodes, have just a UI.Number node with a property called representation where you can choose slider, knob, number, stepper...

Frontend/backend notes

  • A note for the frontend. Should be rendered Markdown and support setting it programmatically
  • A note in the backend. It's canvas wrapped text that's for annotation. Sorely needed for documenting convoluted webs of nodes.

Extensions

Should technically be possible with a rollup workflow

https://stackoverflow.com/questions/65484019/how-can-i-manually-compile-a-svelte-component-down-to-the-final-javascript-and-

As to what can be extended:

  • ComfyUI nodes written in python should be supported with no modifications
  • Add the ability to create new UI controls. A latent space node can be used with a bounding box UI node that outputs X/Y/width/height, and accepts width/height for the total image size
  • Add tabs to the top-level UI just like with webui. Extensions can choose to render whatever they want in those tabs. But I think with ComfyUI being so much more powerful the utility of those tabs will be less so than with webui. Best suited for things like model metadata management and analysis/data extraction of prompts

That should be it I think. Surprisingly little to do with the backend since that's the domain of ComfyUI

X/Y Grid

Strategy:

  1. Extension that can queue multiple prompts at once
  2. Save all prompt IDs for each settings run, watch the prompts for state changes, re-run failed jobs
  3. Combine all the images when prompts have finished running
  4. Can parallelize with multiple GPUs even?

Saner defaults for some ComfyUI backend nodes

Some of the ComfyUI nodes have interesting default ranges for their parameters. Notably the steps and seed parameters have gigantic ranges that make adjusting them with a slider impractical.

So maybe ComfyBox should supply its own defaults for those nodes. And make them user configurable for good measure

Subgraphs/macros

Support saving a chunk of the graph as a litegraph subgraph combined with the ComfyBox panel controls to append to the layout

Checking for correctness is important since the UI being saved needs to be arranged in a modular way

And even if the controls are deeply nested in either the node graph or the UI the layouting system needs to track them still if the parent is updated/deleted (super hard?!)

Improve mobile version

It works but it's pretty barebones. Should have at least the gallery/list/queue sidebar and dark mode added.

Optional nodes per workflow/template

If a node is missing, but declared optional, treat it as NodeMode.NEVER. Also have a new node property called optionalNodeTag that disables the corresponding widgets in the frontend if the node is missing from the install

Inpainting UI

I think klecks is too heavy a dependency for too little benefit. Didn't want to have to mess with canvas drawing myself but I might need to

There should be at least mask editing/zooming/mobile support for now

Link widgets together and copy values

Some widgets should always mirror each other. For example if you lock together TEnc and UNet strength for a LoRA module. There could be two sets of sliders. One has two sliders individually, the other has one slider controlling the other two.

When the locked slider is changed the value should be copied to the other two sliders. But also when the toggle is locked the value should also be copied right then. So that should be handled if a linked widget's visibility changes or through a "bang" action on the checkmark state changing

Since it's a common need could be implemented as a separate feature without the need to use any wires. In the attributes panel you'd be able to set a list of linked widgets that act as senders and receivers. It is a feature in Native Instruments REAKTOR called "IC" (Internal Connection)

Parameter randomization

Useful for discovering new configs for a workflow

Consider that the range of randomization could differ from the raw min/max of the node. For example restricting CFG scale to 3-10 instead of 0-1000000 or whatever. And that the user might want to change these often so they shouldn't be node attributes. Maybe a separate sidebar tab with some preset values dropdown

Global unique nodes

Nodes and links added to subgraphs can have IDs that overlap with the parent graph. This is not good from ComfyBox's perspective since a lot of the data structures assume the ID is unique. It also means the prompt will become invalid on the backend because ComfyUI makes the same assumption. Might havve to replace some of the IDs when a subgraph is attached if they overlap.

Model info sidecar API

A ComfyUI backend extension that adds all the additional networks features missing in base ComfyUI. Browsing models in a database, showing and editing model info, selecting models with an enhanced dropdown.

Expose property modification to frontend

In Max you have the ability to add an attribute as an input slot through the properties panel. For example to toggle visibility on and off. The same should be possible in ComfyBox

And for that matter, exposing the raw value of any arbitrary widget as an input/output parameter. So you can have a toggle that

  • links together more than one set of sliders, like Lora strength
  • hides and shows the correct sliders depending on checked state
  • sets the input slots to the output values of the correct widgets only if it's enabled

Have to be very careful as widget names are assumed to be unique for the purposes of serialization

Decouple widget instances from node instances

Many visual programming languages take this approach. Instead of coupling the UI controls to the node, the two are separated out. There is an explicit slider or button or etc. for every editable parameter.

Makes understanding where sliders are coming from much easier. And you can detach and reattach the two for more control. It's not typical to do in vanilla ComfyUI as the graph canvas itself acts as the UI the majority of the time.

Installation error

I'm in ComfyBox directory in Windows 11, I tried "pnpm install" and get an error:  ERR_PNPM_NO_MATCHING_VERSION_INSIDE_WORKSPACE  In litegraph\packages\core: No matching version found for @litegraph-ts/tsconfig@* inside the workspace

Specify widgets can receive parameters of a type and have presets for them

For example it's pretty common for many workflows to have positive/negative conditioning boxes. So I could tag one with PositivePrompt and then look through my prompt history for something to put into there, so long as the workflow supports a positive/negative prompt.

Requires saving the tag with the widget node but it's doable.

Container types

Tabs, accordion, block, and invisible (no padding/styling)

Block and invisible can be merged with CSS classes use (?)

Accordion should hide the elements with styling instead of removing them in the DOM (?)

Tabs are special since they can have multiple pages. And user might want to toggle them with the graph. But containers won't have litegraph instances bound to them like in REAKTOR since it's not a good design imo. Instead maybe a context menu option for just adding a litegraph node for the tab index

Toggle mode for prompt tags

A lot of times I generate a prompt and the images contain details in the prompt. So they're better picked up by the HR Fix pass I want to add the tags in but turn them off for other prompts.

It would be nice to have a partitioned tags editor like https://github.com/eideehi/sd-webui-better-prompt so tags can be toggled on and off without picking through the textarea

Save snapshots of workspace settings

These will be saved alongside the workflow

For example a quick way of generating small, medium and large images by manipulating parameters all at once

Also a way to apply certain types of snapshots on top and excluding parameters from being saved to the snapshot. Let's you combine different snapshots targeting different features (example: upscale size and Lora model configuration) without combinational explosion

Global event bus

Some events already work in a "global" fashion like the default queue prompt button and the beforeQueued/afterQueued event. So it makes sense to have users create their own global events that are forwarded throughout the graph.

I wonder how this would work however. There would be some global state with a dict of Record<EventName, LGraphNode[]> somewhere I think. When onAction is called each node on the bus is triggered in sequence.

Model info

I want to write a ComfyUI extension that extends the server to send back metadata/info about certain model files.

Node icons

REAKTOR has these, nice for easily telling what type a node is even if its title has been changed

Push/pop workflow settings

Might overlap with #9. But anyway I'll often be prompting a lot and I'll suddenly ask myself "is the LoRA/model I'm using even contributing anything to the final output?" and want to test my theory. But I want to test this on a specific prompt/seed/config that I like so I can see how changing the settings affects only that output.

If I have a bunch of stuff queued already and want to go back to those settings after my tests then it's too hard right now. Instead would be neat to have a settings stack per-workflow that you can push on/pop off or save for later. Kind of like git stash I guess.

Also the saved data could be converted to standard prompts for reuse in other workflows, too.

Different views for desktop and mobile

It needs the ability to display the UI differently depending on mobile or desktop screens. Preferably using media query syntax. Also need the ability to set different attributes depending on the view type.

So you'd have the same nodes but in different places depending on the workflow being shown. And possibly different classes/other attributes for easier mobile use.

Localization

People should be able to add on their own localizations if they have a special workflow. So three things are needed:

  1. Fallback string to use if localization is missing
  2. Localization key for the text
  3. List of keys => text for supported languages saved to the current workflow

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.