Giter VIP home page Giter VIP logo

chainner-org / chainner Goto Github PK

View Code? Open in Web Editor NEW
4.2K 59.0 267.0 31.76 MB

A node-based image processing GUI aimed at making chaining image processing tasks easy and customizable. Born as an AI upscaling application, chaiNNer has grown into an extremely flexible and powerful programmatic image processing application.

Home Page: https://chaiNNer.app

License: GNU General Public License v3.0

Python 49.64% JavaScript 0.97% HTML 0.01% TypeScript 48.65% SCSS 0.73% Shell 0.01%

chainner's Introduction

chaiNNer

GitHub Latest Release GitHub Total Downloads License Discord ko-fi PRs Welcome

A node-based image processing GUI aimed at making chaining image processing tasks easy and customizable. Born as an AI upscaling application, chaiNNer has grown into an extremely flexible and powerful programmatic image processing application.

ChaiNNer gives you a level of customization of your image processing workflow that very few others do. Not only do you have full control over your processing pipeline, you can do incredibly complex tasks just by connecting a few nodes together.

ChaiNNer is also cross-platform, meaning you can run it on Windows, MacOS, and Linux.

For help, suggestions, or just to hang out, you can join the chaiNNer Discord server

Remember: chaiNNer is still a work in progress and in alpha. While it is slowly getting more to where we want it, it is going to take quite some time to have every possible feature we want to add. If you're knowledgeable in TypeScript, React, or Python, feel free to contribute to this project and help us get closer to that goal.

Installation

Download the latest release from the Github releases page and run the installer best suited for your system. Simple as that.

You don't even need to have Python installed, as chaiNNer will download an isolated integrated Python build on startup. From there, you can install all the other dependencies via the Dependency Manager.

If you do wish to use your system Python installation still, you can turn the system Python setting on. However, it is much more recommended to use integrated Python. If you do wish to use your system Python, we recommend using Python 3.11, but we try to support 3.8, 3.9, and 3.10 as well.

If you'd like to test the latest changes and tweaks, try out our nightly builds

How To Use

Basic Usage

While it might seem intimidating at first due to all the possible options, chaiNNer is pretty simple to use. For example, this is all you need to do in order to perform an upscale:

Before you get to this point though, you'll need to install one of the neural network frameworks from the dependency manager. You can access this via the button in the upper-right-hand corner. ChaiNNer offers support for PyTorch (with select model architectures), NCNN, and ONNX. For Nvidia users, PyTorch will be the preferred way to upscale. For AMD users, NCNN will be the preferred way to upscale.

All the other Python dependencies are automatically installed, and chaiNNer even carries its own integrated Python support so that you do not have to modify your existing Python configuration.

Then, all you have to do is drag and drop (or double click) node names in the selection panel to bring them into the editor. Then, drag from one node handle to another to connect the nodes. Each handle is color-coded to its specific type, and while connecting will show you only the compatible connections. This makes it very easy to know what to connect where.

Once you have a working chain set up in the editor, you can press the green "run" button in the top bar to run the chain you have made. You will see the connections between nodes become animated, and start to un-animate as they finish processing. You can stop or pause processing with the red "stop" and yellow "pause" buttons respectively.

Don't forget, there are plenty of non-upscaling tasks you can do with chaiNNer as well!

Tips & Tricks

To select multiple nodes, hold down shift and drag around all the nodes you want to be selected. You can also select an individual node by just clicking on it. When nodes are selected, you can press backspace or delete to delete them from the editor.

To perform batch processing on a folder of images, use the "Load Images" node. To process videos, use the "Load Video" node. It's important to note however that you cannot use both "Load Images" and "Load Video" nodes (or any two nodes that perform batch iteration) together in a chain. You can however combine the output (collector) nodes in the chain, for example using "Save Image" with "Load Video", and "Save Video" with "Load Images".

You can right-click in the editor viewport to show an inline nodes list to select from. You also can get this menu by dragging a connection out to the editor rather than making an actual connection, and it will show compatible nodes to automatically create a connection with.

Helpful Resources

Compatibility Notes

  • MacOS versions 10.x and below are not supported.

  • Windows versions 8.1 and below are also not supported.

  • Apple Silicon Macs should support almost everything. Although, ONNX only supports the CPU Execution Provider, and NCNN sometimes does not work properly.

  • Some NCNN users with non-Nvidia GPUs might get all-black outputs. I am not sure what to do to fix this as it appears to be due to the graphics driver crashing as a result of going out of memory. If this happens to you, try manually setting a tiling amount.

  • To use the Clipboard nodes, Linux users need to have xclip or, for wayland users, wl-copy installed.

GPU Support

For PyTorch inference, only Nvidia GPUs are officially supported. If you do not have an Nvidia GPU, you will have to use PyTorch in CPU mode. This is because PyTorch only supports Nvidia's CUDA. MacOS users on Apple Silicon Macs can also take advantage of PyTorch's MPS mode, which should work with chaiNNer.

If you have an AMD or Intel GPU that supports NCNN however, chaiNNer now supports NCNN inference. You can use any existing NCNN .bin/.param model files (only ESRGAN-related SR models have been tested), or use chaiNNer to convert a PyTorch or ONNX model to NCNN.

For NCNN, make sure to select which GPU you want to use in the settings. It might be defaulting to your integrated graphics!

For Nvidia GPUs, ONNX is also an option to be used. ONNX will use CPU mode on non-Nvidia GPUs, similar to PyTorch.

Model Architecture Support

ChaiNNer currently supports a limited amount of neural network architectures. More architectures will be supported in the future.

PyTorch

As of v0.21.0, chaiNNer uses our new package called Spandrel to support Pytorch model architectures. For a list of what's supported, check out the list there.

NCNN

Single Image Super Resolution

  • Technically, almost any SR model should work assuming they follow a typical CNN-based SR structure. However, I have only tested with ESRGAN (and its variants) and with Waifu2x.

ONNX

Single Image Super Resolution

  • Similarly to NCNN, technically almost any SR model should work assuming they follow a typical CNN-based SR structure. However, I have only tested with ESRGAN.

Background Removal

Troubleshooting

For troubleshooting information, view the troubleshooting document.

Building chaiNNer Yourself

I provide pre-built versions of chaiNNer here on GitHub. However, if you would like to build chaiNNer yourself, simply run npm install (make sure that you have at least npm v7 installed) to install all the nodejs dependencies, and npm run make to build the application.

FAQ

For FAQ information, view the FAQ document.

chainner's People

Contributors

adegerard avatar adodge avatar bigboybarney avatar dependabot[bot] avatar emarron avatar flashlightet avatar harisreedhar avatar illode avatar jabbawack0 avatar jackcarey avatar jeremyrand avatar joeyballentine avatar jumpyjacko avatar justnoon avatar kim2091 avatar mafiosnik777 avatar mateusz avatar mrjschulte avatar pifroggi avatar raysit avatar rundevelopment avatar shipengtaov avatar splendide-imaginarius avatar sryvkver avatar stonerl avatar the-database avatar theflyingzamboni avatar zarxrax avatar zeptofine avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chainner's Issues

Image quality metric node(s)

When testing models I compare the upscaled to the original. being about to get the SSIM and PSNR values for one image compared to the other would be a useful tool and being able to use that in the node system would be super handy.

more of a stretch but if possible a VMAF node would be great too

Add support for branching

We could do branching by using a special value representing a not-taken branch:

The main problem is that the current execution engine takes all possible paths. This is good and what we want it to do, but it makes branching (where we only want it to take some paths) difficult. So how about we use a special value, let's call it D, to represent a branch that is not taken?
To create a dead branch, a node simply has to output D in any of its outputs. Nodes would then handle D inputs as follows: if any of the required inputs is D, output D, and if an optional input is D, set the input to its default value. This is enough to implement branching (assuming no cycles).

This system would transparently implement branching. The D-input check would happen before actually running nodes, so our current nodes will just work in this system.

That being said, this only how to do branching. I don't have any good ideas as for how we could do the condition expressions.

For more information, see here.

Don't save empty views

I have found a bug where saving a workflow will just write an empty workflow to disk:

{"version":"0.5.2","content":{"nodes":[],"edges":[],"viewport":{"x":0,"y":0,"zoom":1}},"timestamp":"2022-04-27T17:36:16.425Z"}

I'm still trying to reproduce that bug and will make a separate issue for it.

In the meantime, please add a warning or make it impossible to save empty workflows. Saving empty workflows has no value anyway and would have made it easier to detect this bug.


Also, why is the JSON base64 encoded? Why not just save plain JSON and make the .cha files easy to debug?

Save notification

Right now, there is no indication that the current chain has been saved successfully. Add a save notification (somewhere) so that the user knows that their chain has been saved (and where to).

Merge Load Image and Iterator Nodes

As it is currently it appears you have to create your chain twice if you want to use it on a specific image and then decide to run a batch. Would be more intuitive and less work to have the iterator node removed and its functions integrated into the regular load image node for when you have a folder selected instead of a single image.
Or if it's less work to program just make the iterator node compatible with existing nodes instead of forcing it into its own empty window.

While I'm already posting I would like to add that implementing file drag&drop functionality to the load image and load model nodes would also be very useful.

[ Question ] Does NCNN node use RealESRGAN or ESRGAN?

I'm very new to upscaling, so apologies if this is a dumb question!

For cupscale, we have the option to use "RealESRGAN" or "ESRGAN" with NCNN. RealESRGAN is faster but only supports 4x models (not sure if it's a limitation of the implementation or RealESRGAN). Which one does chainner use?

DirectML target check

If there is no cuda, do a check if dml is available and if it is then use that as the device

[ Question ] Where are dependencies installed in a portable install?

I chose to install with the portable install of chainner. I had hoped all dependencies would be installed under that portable folder but it doesn't seem to be the case (or I'm missing them).

Can someone confirm where those dependencies are? I know I can uninstall from the managed dependencies folder for some apps but python (integrated) cannot and I was curious where that was (I swear I alread had python installed).

[Bug] 'optional' flag in Text Append node doesn't work

Currently, every field in the Text Append node has to have something entered in order for the flowchart to run, which can be an issue when you only have two text inputs you want to append. Ideally, they should be able to be left blank.

Edit: Looks like this is actually implemented in utility_nodes.py, but it doesn't seem to be working as intended.

Custom nodes?

Is/will there be support for custom nodes?

I have a few things that I need chainner to support for it to be useful to me. I've already implemented these feature in a C# project of mine, but I can easily port them to any language.

Fix blur node

Right now, the blur node is just broken.

  1. It approximates Gaussian blur because the simple blur is iterated 16 times.
  2. The amount input is not a radius (as is the case with Gaussian blur), but the kernel size. So if the amount is even, the kernel will move the whole image by one pixel (times 16).

Fixing either of these problems is a breaking change.

invalid opcode in libtorch_cuda_cpp.so

When I try to do an image upscale the upscaling fails and I see the following two things in the logfiles

Python subprocess exited with code null and signal SIGILL
kernel: [ 7750.737022] traps: python[176224] trap invalid opcode ip:7febea60213e sp:7feb82c920d0 error:0 in libtorch_cuda_cpp.so[7febe27a0000+61c96000]

So this seems to has something to do with my graphic card, which is a RTX3060, 12GB, because using CPU instead of GPU works.

I'm running this under Kubuntu 21.10.
Driver info: Driver Version: 470.103.01 CUDA Version: 11.4

If this is not directly related to chaiNNer, can you please add Information to the documentation, e.g. how to make pytorch to work with these Graphiccards?

I used the latest chaiNNer 0.4.3

Save files with Resize nodes break when PIL is installed

When PIL gets installed, old save files with Resize nodes break.

This is because the interpolation method input uses the literal interpolation method values of either PIL or OpenCV. The those values depend on what you have installed.

I'm currently working on a fix for this. Since the numerical values of PIL and OpenCV are incompatible with each other, I will choose the PIL values since most of our users likely have PIL installed.

Add interpolation slider labels

Label ends of interpolation sliders for PyTorch and NCNN as 'Model A' and 'Model B'. Make sure functions match that ordering.

Many DDS DXT10 formats are not supported

The underlying image library we use does not support many DDS DXT10 formats.

This includes:

  • ATI1 (legacy)
  • ATI2 (legacy)
  • BC1 (all variants)
  • BC7 (all variants)

Disable electron menu and shortcuts while an alert is open

Right now, our custom alert have the problem that they can be undermined. E.g. the "Discard unsaved changes?" alert can be undermined by just opening a new file (see #219) or saving the current file. The same is also true for lots of other alerts.

The basic problem is that the user can still do other inputs by using the menu or shortcuts. This essentially bypasses the input lock imposed by the alert.

Remember window size

Remember window size on close so that the next time chaiNNer opens, it retains the same dimensions.

Node context menus

Right clicking a node should open a context menu similar to the ... button in the node footer, but at the point of right clicking.

This would be best accomplished via a context similar to the alert box context, where a single menu is used instead of multiple instances of it.

Saving does not work correctly

When saving a new workflow via Ctrl+S, the workflow won't be saved.

Steps to reproduce:

  1. Make a new workflow
  2. Add some nodes
  3. Ctrl+S
  4. Choose a file name and save
  5. Open the saved file
  6. All nodes will now be gone

This is because the saved file does not contain any information about the nodes. Example:

{"version":"0.5.2","content":{"nodes":[],"edges":[],"viewport":{"x":305.96171293776086,"y":-174.04899292324444,"zoom":0.5244057339865723}},"timestamp":"2022-04-27T17:50:34.429Z"}

Unsaved changes warning

When closing chaiNNer, creating a new chain, or opening another chn file, pop up a prompt if there are unsaved changes to the current chain and ask whether to save.

[ Feature ] Pytorch Model to NCNN Model

As an AMD user, the automatic conversion from the pytorch model format to ncnn is really nice to have in cupscale. For now I can use that tool and use any converted models in chaiNNER but it would nice to support something similar natively in chaiNNER.

I know this was in the readme but I wanted to capture it as an issue so someone would be more likely to work on it :)

Add a setting to disable hardware acceleration for the GUI

On Win10, the Nvidia 1080 Ti driver has a bug that causes all Electron applications to become blurry.

image

The bug has been present for 4 years now, and I lost all hope of it ever getting fixed. Luckily, there is a simple workaround. Disabling hardware acceleration for the Electron app fixes this issue. This workaround works for Slack, VS Code, and Discord, and it works for chaiNNer as well.

Please add an option in the settings for this, so I don't have to start chaiNNer from the command line with --disable-gpu anymore.

Add documentation to nodes

Right now, nodes only have a short summary of their function.

Add documentation and usage information for all nodes. The current idea is to have a little (i) icon next to the node name that opens a popup with the docs.

Backend: No module named 'nodes'

Trying to run chainner on Windows with Python 3.8.5 but I cannot get past this error.

11:15:12.966 > Attempting to spawn backend...
11:15:12.969 > Successfully spawned backend.
11:15:13.220 > Backend: No module named 'nodes'

11:15:13.221 > Backend: No module named 'nodes'

11:15:13.521 > Backend: No module named 'nodes'

11:15:13.521 > Backend: Traceback (most recent call last):
  File "F:\chainner\resources\backend\run.py", line 37, in <module>
    from nodes.node_factory import NodeFactory
ModuleNotFoundError: No module named 'nodes'

Add "Copy system information to clipboard" button

We often have issues where the devs need to know some system information and the current settings.

Add a little button that automatically collects all this info and copies it to the users clipboard, so the can send it to us.

This should include:

  • All current settings
  • OS
  • CPU/GPU model + specs
  • Environment variables

[ BUG ] Installing NCNN gave error output

Installing NCNN gave an error output about restarting chainner. I did so but NCNN still wasn't installed. I tried again and again but still it wasn't installed.

Finally, I just let it sit and NCNN and the other dependencies were marked as installed. If chainner is installing, it'd be nice to have some indication and not get an error.

Custom preview window

Running the Preview Image node would pop up a custom chakra window. This custom window would have the following features, as shown in the mockup below:

  1. Window should be fully resizable.
  2. Chain should pause while window is open to prevent window spam.
  3. Preview Image should accept multiple image inputs (accept an iterable somehow in potential new iteration system?). All simultaneous inputs (so one set per iteration if chain is iterated) would be opened in different tabs of the Preview Window (as seen in the tab bar in the mockup).
  4. The image displayed in the viewing window is swapped by clicking the appropriate tab or using the corresponding number key. Number keys would be assigned in order. If more than 10 images are permitted, could use key combos with the number keys to provide more hotkeys.
  5. Scroll wheel or zoom buttons in toolbar (and maybe +/- keys) should zoom in and out of the images. Click and drag should pan the images.
  6. Images should be scaled to the image with the largest dimensions (default nearest, but could allow other interpolations selectable from toolbar as well), as all tabbed images should share the same zoom scale and xy offset (that is, if you swap between different images, you should be seeing the same part of the image in the viewing window), as the primary purpose of this feature is to compare upscales of an image.
  7. The Save button would pop up a save as window to save the currently selected image. Save All would save the images in all currently open tabs to a selected directory. Would need to figure out the best way to provide names for all of them, and possibly different behavior depending on whether file names are input to Preview Image. Could use a pattern like the GIMP Export all layers plugin (that plugin defaults to {image-name}-{layer-name}.png, just as an example), or save as the original input file name with the number appended, etc.
  8. If possible: If preview is for a single image chain and not an iterator, have a 'Rerun current view' button that will rerun the chain on a cutout of the original image input based on the current section of the image visible in the viewing window, allowing chain to be rerun on partial images to compare specific sections only without having to process the full array.

Preview_mockup

Per node-type settings

We recently had feedback where a user wanted to set their own defaults for the interpolation method of the resize nodes and the starting directory for image load/save and model load/save. This is definitely a useful feature, especially for power users.

Add settings for each node type that allow the user to override the defaults values/behavior of newly created nodes. Those settings should also be saved in localStorage like regular application settings. The setting should be displayed via a special menu that can be opened by clicking a gear icon on the node type in the node selector. The menu is also per node-type.

Note: These settings must only affect the default values and behaviors. The resulting chain must still be portable.

BoundedIntegerInput results in str

I ran into this issue with BoundedIntegerInput, but all other *IntegerInput classes also yield strings.

Why? The name says int, so why does it pass strings to the run method of nodes?

I went through all currently available nodes and all that use an *IntegerInput either:

  1. Convert the string to int
  2. Convert the string float
  3. Let numpy convert it to float

So why does it pass strings?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.