Giter VIP home page Giter VIP logo

xuniren's Introduction

虚拟人说话头生成

Get Started

Installation

Tested on Ubuntu 22.04, Pytorch 1.12 and CUDA 11.6,or Pytorch 1.12 and CUDA 11.3

git clone https://github.com/waityousea/xuniren.git
cd xuniren

Install dependency

# for ubuntu, portaudio is needed for pyaudio to work.
sudo apt install portaudio19-dev

pip install -r requirements.txt
or
## environment.yml中的pytorch使用的1.12和cuda 11.3
conda env create -f environment.yml 
## install pytorch3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

Build extension (optional)

By default, we use load to build the extension at runtime. However, this may be inconvenient sometimes. Therefore, we also provide the setup.py to build each extension:

# install all extension modules
bash scripts/install_ext.sh

start(独立运行)

环境配置完成后,启动虚拟人生成器:

python app.py

start(对接fay,在ubuntu 20下完成测试)

环境配置完成后,启动fay对接脚本

python fay_connect.py

扫码支助开源开发工作,凭支付单号入qq交流群

接口的输入与输出信息 Websoket.md

虚拟人生成的核心文件

## 注意,核心文件需要单独训练
.
├── data
│   ├── kf.json			
│   ├── pretrained
│   └── └── ngp_kg.pth

Inference Speed

在台式机RTX A4000或笔记本RTX 3080ti的显卡(显存16G)上进行视频推理时,1s可以推理35~43帧,假如1s视频25帧,则1s可推理约1.5s视频。

Acknowledgement

  • The data pre-processing part is adapted from AD-NeRF.
  • The NeRF framework is based on torch-ngp.
  • The algorithm core come from RAD-NeRF.
  • Use example Fay.

学术交流可发邮件到邮箱:[email protected]

xuniren's People

Contributors

waityousea avatar xszyou avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.