llava-vl / llava-interactive-demo Goto Github PK

View Code? Open in Web Editor NEW

344.0 16.0 25.0 1.84 MB

LLaVA-Interactive-Demo

Home Page: https://llava-vl.github.io/llava-interactive/

License: Apache License 2.0

Dockerfile 2.04% Python 93.92% Shell 4.03%

lmm multimodal

llava-interactive-demo's Introduction

🌋 LLaVA-Interactive

An All-in-One Demo for Image Chat, Segmentation and Generation/Editing.

[Project Page] ~~[Demo]~~ [Paper]

⚠️ As of Jun 10, 2024 the live demo or playground website is disabled.

Install

Installing this project requires CUDA 11.7 or above. Follow the steps below:

git clone https://github.com/LLaVA-VL/LLaVA-Interactive-Demo.git
conda create -n llava_int -c conda-forge -c pytorch python=3.10.8 pytorch=2.0.1 -y
conda activate llava_int
cd LLaVA-Interactive-Demo
pip install -r requirements.txt
source setup.sh

Run the demo

To run the demo, simply run the shell script.

./run_demo.sh

Citation

If you find LLaVA-Interactive useful for your research and applications, please cite using this BibTeX:

  @article{chen2023llava_interactive,
    author      = {Chen, Wei-Ge and Spiridonova, Irina and Yang, Jianwei and Gao, Jianfeng and Li, Chunyuan},
    title       = {LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing},
    publisher   = {arXiv:2311.00571},
    year        = {2023}
  }

Related Projects

Acknowledgement

LaMa: A nice tool we use to fill the background holes in images.

Terms of use

By using this service, users are required to agree to the following terms: The service is a research preview intended for non-commercial use only. It only provides limited safety measures and may generate offensive content. It must not be used for any illegal, harmful, violent, racist, or sexual purposes. The service may collect user dialogue data for future research. For an optimal experience, please use desktop computers for this demo, as mobile devices may compromise its quality.

License

This project including LLaVA and SEEM are licensed under the Apache License. See the LICENSE file for more details. The GLIGEN project is licensed under the MIT License.

The service is a research preview intended for non-commercial use only, subject to the model License of LLaMA, Terms of Use of the data generated by OpenAI, and Privacy Practices of ShareGPT. Please contact us if you find any potential violation.

llava-interactive-demo's People

Contributors

Stargazers

Watchers

llava-interactive-demo's Issues

Where can I find module: 'GuardlistPython'?

LLaVA-Interactive-Demo/./LLaVA/llava/guardlistWrapper.py:3 in │
│ │
│ │
│ 1 # this import, even if not directly used, will take care of correctly setting the p │
│ 2 # see init.py in package │
│ ❱ 3 import GuardlistPython │
│ 4 │
│ 5 from Microsoft.Office.Guardlist.Client import (EnvironmentType, │
│ 6 │ │ │ │ │ │ │ │ │ │ │ GuardlistInput, │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'GuardlistPython'

When I run llava_interactive.py as README.md，
python llava_interactive.py
--moderate
input_text_guardlist
input_text_aics
input_image_aics
output_text_guardlist
output_text_aics
gligen_input_text_guardlist
gligen_input_text_aics
gligen_output_image_aics

I can't find Module 'GuardlistPython' and 'Microsoft.Office.Guardlist.Client' in the project file, and I try pip install also can't find the match moudle

I followed the instructions but it gave me error messages, please help!

https://github.com/LLaVA-VL/LLaVA-Interactive-Demo?tab=readme-ov-file

Serving Flask app 'lama_server' (lazy loading)
Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
Debug mode: on
Running on http://127.0.0.1:9171/ (Press CTRL+C to quit)
Restarting with stat
Detectron v2 is not installed
/home/sagemaker-user/LLaVA-Interactive-Demo/lama
Debugger is active!
Debugger PIN: 408-623-478
/home/sagemaker-user/LLaVA-Interactive-Demo
Traceback (most recent call last):
File "/home/sagemaker-user/LLaVA-Interactive-Demo/llava_interactive.py", line 7, in
import cv2
File "/home/sagemaker-user/.conda/envs/llava_int/lib/python3.10/site-packages/cv2/init.py", line 181, in
bootstrap()
File "/home/sagemaker-user/.conda/envs/llava_int/lib/python3.10/site-packages/cv2/init.py", line 153, in bootstrap
native_module = importlib.import_module("cv2")
File "/home/sagemaker-user/.conda/envs/llava_int/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libGL.so.1: cannot open shared object file: No such file or directory

Demo is down

As shown below:

When use run_demo.sh, it outputs'"GET / HTTP/1.1" 404 Not Found'

2023-11-03 10:40:36 | INFO | stdout | INFO: 127.0.0.1:14124 - "GET / HTTP/1.1" 404 Not Found

EnvironmentNameNotFound: Could not find conda environment: lama
You can list all discoverable environments with conda info --envs.

/LLaVA-Interactive-Demo/lama
Traceback (most recent call last):
File "/LLaVA-Interactive-Demo/lama/../lama_server.py", line 7, in
import hydra
ModuleNotFoundError: No module named 'hydra'
2023-11-03 10:41:06 | INFO | stdout | INFO: 127.0.0.1:14122 - "GET / HTTP/1.1" 404 Not Found
Traceback (most recent call last):
File "/LLaVA-Interactive-Demo/llava_interactive.py", line 7, in
import cv2
ModuleNotFoundError: No module named 'cv2'

And there is no conda env named 'lama', maybe I missed something in setup? But I just followed the README
I run run_demo.sh in 'llava_int' env created in setup.

thaks

| ERROR | stderr | KeyError: 'CONTENT_SAFETY_ENDPOINT'

how to solve the problem of "| ERROR | stderr | KeyError: 'CONTENT_SAFETY_ENDPOINT'" and "KeyError: '"CONTENT_SAFETY_KEY"'". Please tell me what should I set these two variables to.

UnavailableInvalidChannel: HTTP 404 NOT FOUND for channel conda_forge <https://conda.anaconda.org/conda_forge>

conda create -n llava_int -c conda_forge python=3.10.8 pytorch=2.0.1

Run the above command line yields the errors:

Collecting package metadata (current_repodata.json): failed

UnavailableInvalidChannel: HTTP 404 NOT FOUND for channel conda_forge <https://conda.anaconda.org/conda_forge>

The channel is not accessible or is invalid.

You will need to adjust your conda configuration to proceed.
Use `conda config --show channels` to view your configuration's current state,
and use `conda config --show-sources` to view config file locations.