openvtuber's Introduction

OpenVtuber-虚拟爱抖露共享计划

运行方法（Easy Start）

node ./NodeServer/server.js
make -C ./PythonClient/rcnn/
python3.7 ./PythonClient/vtuber_usb_camera.py --gpu -1

人脸检测（Face Detection）

RetinaFace is a practical single-stage SOTA face detector which is initially described in arXiv technical report

头部姿态估计（Head Pose Estimation）

head-pose-estimation

特征点检测（Facial Landmarks Tracking）

Download Landmarks Model
Using dlib for facial features tracking
Algorithm from CVPR 2014
Training set is based on i-bug 300-W datasets. It's annotation is shown below:

表情识别（Emotion Recognition）

face_classification
IMDB gender classification test accuracy: 96%.
fer2013 emotion classification test accuracy: 66%.

Live2D

推流框架

MJPEG Framework

FAQ

Why use RetinaFace ?

Methods LFW CFP-FP AgeDB-30

MTCNN+ArcFace 99.83 98.37 98.15

RetinaFace+ArcFace 99.86 99.49 98.60

Methods	LFW	CFP-FP	AgeDB-30
MTCNN+ArcFace	99.83	98.37	98.15
RetinaFace+ArcFace	99.86	99.49	98.60

Citation

@article{7553523,
    author={K. Zhang and Z. Zhang and Z. Li and Y. Qiao}, 
    journal={IEEE Signal Processing Letters}, 
    title={Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks}, 
    year={2016}, 
    volume={23}, 
    number={10}, 
    pages={1499-1503}, 
    keywords={Benchmark testing;Computer architecture;Convolution;Detectors;Face;Face detection;Training;Cascaded convolutional neural network (CNN);face alignment;face detection}, 
    doi={10.1109/LSP.2016.2603342}, 
    ISSN={1070-9908}, 
    month={Oct}
}
  
@inproceedings{deng2019retinaface,
title={RetinaFace: Single-stage Dense Face Localisation in the Wild},
author={Deng, Jiankang and Guo, Jia and Yuxiang, Zhou and Jinke Yu and Irene Kotsia and Zafeiriou, Stefanos},
booktitle={arxiv},
year={2019}
}

Recommend Projects