Anime Style Transfer
Thank you @AK
Anime Style Transfer with Pytorch
- YOLOv5
- Waifu2x
- CLIP
- StyleGAN2(Blending)
- pix2pixHD
Web Demo
Integrated into Huggingface Spaces 🤗 using Gradio. Try out the Web Demo:
My Env
- python 3.7
- pytorch 1.8.1
- cuda 11.1
- cudnn 8.0.5
collect video and image
->
face detection
->
image filtering using clip (clear / blurred)
->
waifu upsampling
->
face alignment according to ffhq standard
->
train Anime StyleGAN (transfer learning!!)
->
FFHQ, Anime StyleGAN Blending
->
Make Dataset
->
Pix2PixHD Training
->
Result !!
1. Prepare custom data
# video
ffmpeg -i video.mp4 -filter:v fps=0.5 video%d.jpg
# folder
anime
|- 1.png
|- 2.png
|- 3.png
|- ...
2. Face Detection with YOLOv5
Custom Training with icartoonface dataset
-
icartoonface format -> yolov5 format
- Reference :
icartoonface_convert.ipynb
- Reference :
-
train yolov5
-
detect + crop
count = 0
for name in tqdm(os.listdir(folder)):
img_path = os.path.join(folder, name)
img = cv2.imread(img_path)
img_h, img_w, _= img.shape
results = model(img_path)
for rst in results.crop():
xmin, ymin, xmax, ymax = rst['box']
xmin = int(xmin.cpu().numpy() * 0.92)
xmax = int(xmax.cpu().numpy() * 1.08)
ymin = int(ymin.cpu().numpy() * 0.92)
ymax = int(ymax.cpu().numpy() * 1.08)
if (xmax - xmin) > (ymax - ymin):
ymax += (((xmax - xmin) - (ymax - ymin)) // 2)
ymin -= (((xmax - xmin) - (ymax - ymin)) // 2)
else:
xmax += (((ymax - ymin) - (xmax - xmin)) // 2)
xmin -= (((ymax - ymin) - (xmax - xmin)) // 2)
xmax = min(xmax, img_w)
ymax = min(ymax, img_h)
xmin = max(xmin, 0)
ymin = max(ymin, 0)
cv2.imwrite(f'./result/{count}.png', img[ymin:ymax, xmin:xmax])
count += 13. Remove Blurred Images with CLIP
["clear face", "blurred face"]4. Image Super-Resoultion with Waifu2x
5. Face Alignment according to FFHQ standard
6. Train StyleGAN2
python dataset_tool.py --source=[YOUR DATASET PATH] --dest=./anime.zip --width 512 --height 512
python train.py --outdir=./training-runs --data=./anime.zip --cfg=paper512 --mirror=1 --gpus=4 --batch 8 --resume ffhq5127. Blend Model
-
Reference :
./blending.ipynb -
Number of Images : 10000
8. Pix2Pix Training
# train
python train.py --name anime --dataroot ./datasets/anime --loadSize 512 --label_nc 0 --no_instance
# test
python test.py --name anime --dataroot ./datasets/test --loadSize 512 --label_nc 0 --no_instance
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent