openmodeldb / open-model-database Goto Github PK

I added support for using chainner compatible upscaling / restoration models to my CLI stable diffusion tool recently. It is a batch generation tool focused on mass generation of images, and automating image generation tasks via scripting, which is used entirely from the command line.

The models from OpenModelDB can be used as a post-processor or pre-processor for diffusion based image generation, or standalone on images, animated images, and videos.

It is implemented in a generic way along side various other image processors in the command line tool which can be used in a similar manner.

It uses tiled upscaling similar to ComfyUI, the code for this is adapted from chainner

I saw on the website a listing of compatible software, and to create a ticket.

I figured it might be of interest, I have been working on this for fun :)

https://github.com/Teriks/dgenerate

Some example uses with the upscaler image processor:

https://github.com/Teriks/dgenerate/tree/v3.0.2?tab=readme-ov-file#upscaling-with-chainner-compatible-upscaler-models

https://github.com/Teriks/dgenerate/blob/v3.0.2/examples/stablediffusion_xl/postprocessors/upscaler-config.txt

https://github.com/Teriks/dgenerate/blob/v3.0.2/examples/config_directives/image_process/upscaler-config.txt

Latest Windows Installer:

https://github.com/Teriks/dgenerate/releases/tag/v3.0.2

Make it so that all model properties can be edited from model pages.

• Name
• Description
• Author (partially)
• Adding and removing authors
• Images
• Download links
• Tags
• Other metadata

This additional little text can provide additional information and context.

Model collections are lists of models with a specific theme. Collections might be:

• A family of models all trained from the same dataset but with different size variants.
• Different iterations of a model.
• A list of models like by a user (favorites).

A collection consists of the following data:

• Its name.
• A de-duplicated list of model IDs that specifies the models in the collection.
• An optional description.
• A list of user IDs that specifies the authors/contributors of the collection.

We need a page that adds models to the db.

Duplicated ones i found:

• Dinjerr/Dinnjerr
• EULA/end user license agreement

Incorrect:

• system-md/blackscout should just be blackscout

https://upscale.wiki/wiki/Official_Research_Models

Add an option to sort search results by:

• Date. As requested on Discord.
• Scale. I think this would be useful.
• Model size. Since model size predicts inference speed, this might be useful.

Add all models linked here: https://github.com/chaiNNer-org/chaiNNer#single-image-super-resolution

We already generate thumbnails for model cards, and I think we should do the same for image our image carousel. Right now, we have some model pages with 10 or more 4k images that all load at once, just to be used as 48x48px thumbnails. This not only increases load times, it's also a performance problem on mobile and lower-end devices.

I think we should generate thumbnails 150% (72px) or 200% (96px) to have sharp images of mobile and high-dpi devices. We can then heavily JPEG-compress them to make them small. They are just thumbnails, so compression artifacts do not matter.

Store the current search query and search tags in the search page URL to make it possible to link to searches.

This is a requirement for #108. It's also useful to share searches.

This should make navigating the website much more efficient for users

Discussed in #246

Make sure to do SEO and that the website has beautiful embeds on services like Discord.

Example: https://openmodeldb.info/models/2x-VimeoScale-Unet

Add all HDCube models. Their license is kinda weird though:

You may not rent, sell or redistribute the generative model or any portion thereof without permission.

(Highlighting by me)

This prevents us from hosting the model ourselves or creating any mirrors.

https://github.com/Venomalia/HDcube

Some models don't have complete data. E.g. a few models don't have input and output channel information, others have scale 0. Basically, clean up invalid data.

We can probably just copy how civit.ai does it, and have a little ? you hover over to see hash info

neither a1111 nor cupscale can use these models. how to use ?

example model:
https://openmodeldb.info/models/1x-AnimeUndeint-Compact

Add a way to clear all tags.

User and/or site owner ratings.

A way to get the good upscalers towards the top of the list, so it's easier to find ones to try first.

It should also include sorting by rating by default.

I removed it for now because those are a lot of model and I don't feel qualified enough to add good description for each.

Thank you very much for sharing this super-resolution model . I tried it and found it effective. However, when I tried to infer the provided fp16 ONNX model, the inference result turned out to be NaN. I wonder if there is any issue with the model (I can get valid results using FP32). Thank you very much.

this model https://openmodeldb.info/models/4x-eula-digimanga-MiA-65k

Show tag descriptions as tooltips in the tag selector.

Hello, thank you very much for your great work! I also want to start training my own super-resolution model. I would like to ask how many high-quality images are needed to train a good model (such as esrgan)? And what kind of graphics card is needed for how long (I have a 3060, may need to rent a 3090)? Is there any recommended training script? Thank you.

We want to allow people to edit models directly on the deployed website, but we need to build the technical foundation first.

The goal of this issue to provide a local editing mode. Users should be able to enter and exit this editing mode at will. While in editing mode, the website will use an in-memory version of the backend database that it will continuously update as the user makes changes. To then share those changes, it also needs to be able to generate a diff (=which files changed in what way).

Implement a tag selector that allows users to include and exclude tags.

While NCNN is technically listed in some places, you can't actually add an NCNN file pair as a resource on a model page, only Pytorch and ONNX. Conversion isn't terribly difficult if you know which tools to use, but it would be nice to directly list these.

Add a little button that allows users to compare images in sort of full screen mode. This may be implemented as actual full screen or just the whole document viewport.

• 1x-Frankendata-FullDegradation-SRFormer is a 4x model.

This is something I've already been working on but now have to kinda redo.

I plan grabbing this data on the backend/next side during build rather than expecting that the image links will always stay up to date.

Info i plan on storing with users:

• Discord ID

Info I will grab using the discord API

• Avatar
• Banner
• Nickname/discriminator

Add a section for similar models to all model pages. 2 models are similar if they have roughly the same functionality.

The similarity metric should mostly be based on tags (with rare tags contributing more), input type, color mode, and scale (maybe more).

Right now, the header search bar/icon is not very good. It doesn't have any feedback in desktop mode, and it's simply a link to the homepage is mobile mode.

I think we should pretty much copy what civitai does. Example: https://civitai.com/models/116225/4x-ultrasharp

As per my suggestion on Discord, this would involve automatically generating thumbnails for each comparison at a lower resolution. This should greatly reduce load on CPU and RAM while scrolling through search results, and should also improve loading speed.

We talked about Lite, Compact, and so on models before and how we could generate tags for that from model sizes. Add support for automatic weight tagging my adding derived tags for it.

We currently have 34 models without tags. We should reduce that number.

I'm on my laptop with 125% DPI scaling and our model cards don't look good.

We should probably detect DPI scaling/zoom in and make sure that we always scale model card images to a whole integer multiple of the pixel scale. So e.g. for 125% DPI scaling (or zoom), we could scale our images 0.8 to get to 1 image pixel = 1 screen pixel or scale 1.6 to get 1 image pixel = 2x2 screen pixels.

Scale 0.8:

Scale 1.6:

When clicking on a tag in model cards or model pages, it should open the search page searching for that tag.

We need a proper license input for model.

Should a small notification toast after updating search results. This gives users feedback on what's going on.

Add a button that brings the user back to the top of the search page.

Even with such a new technology, models quickly become superseeded and improved versions are uploaded.
Hence it's useful to check which of several similarly named models is newer.
I'd kindly request a "date" or "newest/oldest" entry in the sort dropdown menu.

Add a bot that extracts the JSON of models from code blocks in GH issues and then adds them to the database when they are valid and approved. This will allow people to make changes to the project without ever checking out the code or making a PR.

The bot should work as follows:

1. When someone makes an issue, it will scan the issue's body for a special code block containing the diff for the model DB. If none at found, the bot won't do anything.
2. The bot will exact the diff and validate it. Validation works by applying the diff to a CI workflow-local version of the database and then running the usual DB validation. (Validation has to be re-run every time the issue comment is edited and every time the database is changed.)
   1. If checks fail, the bot will post a comment (or update its previous one) explaining the problems validation found. The bot will add a "Needs changes" label to the issue and remove any other bot-related labels.
   2. If checks succeed, the bot will post a comment (or update its previous one) saying that the change is ready to be applied. The bot will add a "Needs review" label to the issue and remove any other bot-related labels.
3. A human reviewer can now review the changes and either request further changes or accept the change. If the reviewer accepts the changes, they will add the "Accepted" label to the issue.
4. The bot listens for label changes and sees the "Accepted" label. It will run validation once more (just to be sure), and upon success it will make a commit directly into the main branch with the changes applied. The issue creator will be tagged as the co-author of the commit.

Having good validation will be important. Ideally, we will get validation to the point where it covers all (simple) mistakes people make.

This file is incomplete, I think. Check that the compatibility data is correct and fill in any gaps.

Compatibility means that any model of that architecture can be converted to the compatible platforms.

When changing tag selection from some tag with few models to no selection (meaning all models are selected), the search page lags, because it has to render cards for all models. The issue here isn't React. Adding that many DOM elements just takes some time.

Find a way to prevent this lag.

There are multiple ways this could be done. Some ideas:

• Pagination.
• Infinite scroll. Display more cards as the user scrolls.
• Spread out adding cards over multiple ticks. So for some tick rate (e.g. 20Hz), render N cards on tick 1, then 2*N cards in tick 2, 3*N cards in tick 3, and so on until all cards are rendered.