RnD Codec
This repository is the implementation of Deep architecture for H.264 decoding.
To install requirements:
pip install -r requirements.txt
To download dataset: tar.gz
This dataset is also accessible on Hugging Face, under the name BotniVision/h264_images.
Data preprocess:
- untar the dataset
- open h264/src/utils/cleanup_samples.py, make sure the dataset folders are correct
- run:
python3 h264/src/utils/cleanup_samples.py
To train the model(s) in the paper, run this command:
python3 h264/src/stages/train_h264_ae_dalle.py --config params_bytes_ae_mssiml1_vgg.yaml --basedir h264
!You need to enter the wandb password.
To generate samples on Test Set, run:
python h264/src/stages/eval_h264_ae_dalle.py --config h264/params_bytes_ae_eval.yaml --basedir h264
!You can configure test samples in config.yaml/dataset/traintest, config.yaml/dataset/validtest or dataset.csv.
You can download pretrained models here:
- model 1 trained 800 epochs on 1.8k(duplicated samples) dataset h264_v20231127.
- model 2 trained 500 epochs on 1.3k dataset h264_v20231127.
- model 3 trained 250 epochs on 1M dataset h264_v20231127, h264_v20240206_1, h264_v20240206_2, h264_v20240206_3, h264_v20240206_4, h264_v20240206_5, h264_v20240206_6, h264_v20240206_7.
Our model achieves the following performance on :
Model name | epochs | Training Loss | Validation Loss | Dataset |
---|---|---|---|---|
model 1 | 800 | 7.93 | 30.632 | 1.8k(duplicated samples) |
model 2 | 500 | 12.863 | 30.448 | 1.3k |
model 3 | 250 | 20.03 | 27.348 | 1M |
#Related Work
https://github.com/apple/ml-cvnets
https://github.com/AntixK/PyTorch-VAE/blob/master/models/mssim_vae.py#L182
https://github.com/crowsonkb/vgg_loss