Giter VIP home page Giter VIP logo

pywaifu's Introduction

PyWaifu: Virtual Anime Waifu Interaction Pipeline

Simple UI web based version chatwaifu

Description:

PyWaifu is an all-in-one pipeline designed to facilitate seamless interactions with virtual anime waifus through Text-to-Speech (TTS), language modeling, and translation capabilities. The primary objective of this project is to enable users to engage in immersive and lifelike conversations with their favorite virtual anime waifus. With PyWaifu, users can generate natural-sounding speech from text inputs, understand the emotional expressions of generated text, easily customize the system, and even translate their conversations into different languages. This unique integration of TTS, language modeling, and translation technologies brings a whole new level of interactivity and enjoyment to the anime fandom, providing users with an unforgettable experience of communicating with their cherished virtual companions.

PyWaifu runs offline on your PC and requires at least 8gb of RAM (in CPU mode).

The anime character illustrations used in this project are provided by @Zr6Ov through Picrew.
Twitter: @Zr6Ov
Picrew Profile Link: ✦絢瀬

*Sound on 🔊

demo.mp4

Features:

  • Text-to-Speech (TTS) conversion
  • Emotion understanding: PyWaifu can analyze the emotional expressions conveyed in the generated text, allowing for a more engaging and dynamic interaction.
  • Customizability: Easily customize and personalize the PyWaifu system according to your preferences.
  • Translation capabilities: Translate conversations with your virtual waifu into different languages, broadening the reach and accessibility of your interactions.
  • User-friendly interface: PyWaifu provides a graphical user interface (GUI) for easy interaction
  • Offline: PyWaifu runs locally on your PC, without the need for an internet connection.

Requirements:

  • Git
  • Python 3.9.16 (recommended)

Installation:

  • Make sure you have installed C/C++ build tools and have CMake installed.
  • Set up a virtual environment (venv) or install Miniconda (optional but strongly recommended).
  • Clone the repository:
    git clone https://github.com/andri-jpg/PyWaifu
    cd PyWaifu
  • Install the required packages:
    pip install -r requirements.txt
    

Usage:

  • Run main.py:
    # For the default mode
    python main.py
    
    # For Indonesian mode
    python main_indonesian.py

Note: You will be prompted to download the suggested model when running main.py. It is recommended to enter 'y' for all questions if you are unsure.

  • The PyWaifu window will appear.
  • You can now enter your input in the text box.
  • You can change the VITS model by replacing the herta.pth and config.json files in the model folder.

Todo:

  • Implement threading and optimize the code for better performance.
  • Polish the graphical user interface (GUI) to enhance the visual appeal and user experience.
  • Add more customization options to allow users to personalize the application according to their preferences.
  • Incorporate animation into the GUI to make it more dynamic and engaging.
  • Create an executable binary of the project using PyInstaller for easy distribution and deployment.

Your feedback is valuable and contributions through pull requests are welcomed to make this project even better.

Star History Chart

Credits:

pywaifu's People

Contributors

andri-jpg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

pywaifu's Issues

OSError: opus-mt-en-id is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

E:\etc\Anaconda\lib\site-packages\transformers\modeling_utils.py:429: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
with safe_open(checkpoint_file, framework="pt") as f:
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): huggingface.co:443
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /opus-mt-en-id/resolve/main/config.json HTTP/1.1" 401 0
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ E:\etc\Anaconda\lib\site-packages\huggingface_hub\utils_errors.py:259 in hf_raise_for_status │
│ │
│ 256 │ │
│ 257 │ """ │
│ 258 │ try: │
│ ❱ 259 │ │ response.raise_for_status() │
│ 260 │ except HTTPError as e: │
│ 261 │ │ error_code = response.headers.get("X-Error-Code") │
│ 262 │
│ │
│ E:\etc\Anaconda\lib\site-packages\requests\models.py:1021 in raise_for_status │
│ │
│ 1018 │ │ │ ) │
│ 1019 │ │ │
│ 1020 │ │ if http_error_msg: │
│ ❱ 1021 │ │ │ raise HTTPError(http_error_msg, response=self) │
│ 1022 │ │
│ 1023 │ def close(self): │
│ 1024 │ │ """Releases the connection back to the pool. Once this method has been │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/opus-mt-en-id/resolve/main/config.json

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ E:\etc\Anaconda\lib\site-packages\transformers\utils\hub.py:409 in cached_file │
│ │
│ 406 │ user_agent = http_user_agent(user_agent) │
│ 407 │ try: │
│ 408 │ │ # Load from URL or cache if already cached │
│ ❱ 409 │ │ resolved_file = hf_hub_download( │
│ 410 │ │ │ path_or_repo_id, │
│ 411 │ │ │ filename, │
│ 412 │ │ │ subfolder=None if len(subfolder) == 0 else subfolder, │
│ │
│ E:\etc\Anaconda\lib\site-packages\huggingface_hub\utils_validators.py:118 in _inner_fn │
│ │
│ 115 │ │ if check_use_auth_token: │
│ 116 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=ha │
│ 117 │ │ │
│ ❱ 118 │ │ return fn(*args, **kwargs) │
│ 119 │ │
│ 120 │ return _inner_fn # type: ignore │
│ 121 │
│ │
│ E:\etc\Anaconda\lib\site-packages\huggingface_hub\file_download.py:1195 in hf_hub_download │
│ │
│ 1192 │ if not local_files_only: │
│ 1193 │ │ try: │
│ 1194 │ │ │ try: │
│ ❱ 1195 │ │ │ │ metadata = get_hf_file_metadata( │
│ 1196 │ │ │ │ │ url=url, │
│ 1197 │ │ │ │ │ token=token, │
│ 1198 │ │ │ │ │ proxies=proxies, │
│ │
│ E:\etc\Anaconda\lib\site-packages\huggingface_hub\utils_validators.py:118 in _inner_fn │
│ │
│ 115 │ │ if check_use_auth_token: │
│ 116 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=ha │
│ 117 │ │ │
│ ❱ 118 │ │ return fn(*args, **kwargs) │
│ 119 │ │
│ 120 │ return _inner_fn # type: ignore │
│ 121 │
│ │
│ E:\etc\Anaconda\lib\site-packages\huggingface_hub\file_download.py:1541 in get_hf_file_metadata │
│ │
│ 1538 │ │ proxies=proxies, │
│ 1539 │ │ timeout=timeout, │
│ 1540 │ ) │
│ ❱ 1541 │ hf_raise_for_status(r) │
│ 1542 │ │
│ 1543 │ # Return │
│ 1544 │ return HfFileMetadata( │
│ │
│ E:\etc\Anaconda\lib\site-packages\huggingface_hub\utils_errors.py:291 in hf_raise_for_status │
│ │
│ 288 │ │ │ │ " repo_type.\nIf you are trying to access a private or gated repo," │
│ 289 │ │ │ │ " make sure you are authenticated." │
│ 290 │ │ │ ) │
│ ❱ 291 │ │ │ raise RepositoryNotFoundError(message, response) from e │
│ 292 │ │ │
│ 293 │ │ elif response.status_code == 400: │
│ 294 │ │ │ message = ( │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-64aa705b-6d4ca9f10008f9c02f966fa7)

Repository Not Found for url: https://huggingface.co/opus-mt-en-id/resolve/main/config.json.
Please make sure you specified the correct repo_id and repo_type.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ E:\etc\PyWaifu\main.py:29 in │
│ │
│ 26 ) │
│ 27 │
│ 28 ts = tts_infer(model_name=configs['vits_model']) │
│ ❱ 29 tl = translator(indonesian=True) │
│ 30 │
│ 31 │
│ 32 pygame.init() │
│ │
│ E:\etc\PyWaifu\translate.py:49 in init
│ │
│ 46 │ │ │ self.model_id = MarianMTModel.from_pretrained("opus-mt-id-en") │
│ 47 │ │ │ self.tokenizer_id = AutoTokenizer.from_pretrained("opus-mt-id-en") │
│ 48 │ │ │ self.model_id = self.model_id.to(self.device) │
│ ❱ 49 │ │ │ self.model_en = MarianMTModel.from_pretrained("opus-mt-en-id") │
│ 50 │ │ │ self.tokenizer_en = AutoTokenizer.from_pretrained("opus-mt-en-id") │
│ 51 │ │ │ self.model_en = self.model_en.to(self.device) │
│ 52 │
│ │
│ E:\etc\Anaconda\lib\site-packages\transformers\modeling_utils.py:2250 in from_pretrained │
│ │
│ 2247 │ │ # Load config if we don't provide a configuration │
│ 2248 │ │ if not isinstance(config, PretrainedConfig): │
│ 2249 │ │ │ config_path = config if config is not None else pretrained_model_name_or_pat │
│ ❱ 2250 │ │ │ config, model_kwargs = cls.config_class.from_pretrained( │
│ 2251 │ │ │ │ config_path, │
│ 2252 │ │ │ │ cache_dir=cache_dir, │
│ 2253 │ │ │ │ return_unused_kwargs=True, │
│ │
│ E:\etc\Anaconda\lib\site-packages\transformers\configuration_utils.py:546 in from_pretrained │
│ │
│ 543 │ │ assert config.output_attentions == True │
│ 544 │ │ assert unused_kwargs == {"foo": False} │
│ 545 │ │ ```""" │
│ ❱ 546 │ │ config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwarg │
│ 547 │ │ if "model_type" in config_dict and hasattr(cls, "model_type") and config_dict["m │
│ 548 │ │ │ logger.warning( │
│ 549 │ │ │ │ f"You are using a model of type {config_dict['model_type']} to instantia │
│ │
│ E:\etc\Anaconda\lib\site-packages\transformers\configuration_utils.py:573 in get_config_dict │
│ │
│ 570 │ │ """ │
│ 571 │ │ original_kwargs = copy.deepcopy(kwargs) │
│ 572 │ │ # Get config dict associated with the base config file │
│ ❱ 573 │ │ config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwar │
│ 574 │ │ if "_commit_hash" in config_dict: │
│ 575 │ │ │ original_kwargs["_commit_hash"] = config_dict["_commit_hash"] │
│ 576 │
│ │
│ E:\etc\Anaconda\lib\site-packages\transformers\configuration_utils.py:628 in _get_config_dict │
│ │
│ 625 │ │ │ │
│ 626 │ │ │ try: │
│ 627 │ │ │ │ # Load from local folder or from cache or download from model Hub and ca │
│ ❱ 628 │ │ │ │ resolved_config_file = cached_file( │
│ 629 │ │ │ │ │ pretrained_model_name_or_path, │
│ 630 │ │ │ │ │ configuration_file, │
│ 631 │ │ │ │ │ cache_dir=cache_dir, │
│ │
│ E:\etc\Anaconda\lib\site-packages\transformers\utils\hub.py:424 in cached_file │
│ │
│ 421 │ │ ) │
│ 422 │ │
│ 423 │ except RepositoryNotFoundError: │
│ ❱ 424 │ │ raise EnvironmentError( │
│ 425 │ │ │ f"{path_or_repo_id} is not a local folder and is not a valid model identifie │
│ 426 │ │ │ "listed on 'https://huggingface.co/models'\nIf this is a private repository, │
│ 427 │ │ │ "pass a token having permission to this repo with use_auth_token or log in │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OSError: opus-mt-en-id is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.
(base) PS E:\etc\PyWaifu>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.