RnD Codec

This repository is the implementation of Deep architecture for H.264 decoding.

To install requirements:

pip install -r requirements.txt

To download dataset: tar.gz

This dataset is also accessible on Hugging Face, under the name BotniVision/h264_images.

Data preprocess:

• untar the dataset
• open h264/src/utils/cleanup_samples.py, make sure the dataset folders are correct
• run:

python3 h264/src/utils/cleanup_samples.py

To train the model(s) in the paper, run this command:

python3 h264/src/stages/train_h264_ae_dalle.py --config params_bytes_ae_mssiml1_vgg.yaml --basedir h264

!You need to enter the wandb password.

To generate samples on Test Set, run:

python h264/src/stages/eval_h264_ae_dalle.py --config h264/params_bytes_ae_eval.yaml --basedir h264

!You can configure test samples in config.yaml/dataset/traintest, config.yaml/dataset/validtest or dataset.csv.

You can download pretrained models here:

• model 1 trained 800 epochs on 1.8k(duplicated samples) dataset h264_v20231127.
• model 2 trained 500 epochs on 1.3k dataset h264_v20231127.
• model 3 trained 250 epochs on 1M dataset h264_v20231127, h264_v20240206_1, h264_v20240206_2, h264_v20240206_3, h264_v20240206_4, h264_v20240206_5, h264_v20240206_6, h264_v20240206_7.

Our model achieves the following performance on :

#Related Work

https://github.com/apple/ml-cvnets

https://github.com/AntixK/PyTorch-VAE/blob/master/models/mssim_vae.py#L182

https://github.com/crowsonkb/vgg_loss

https://github.com/psyrocloud/MS-SSIM_L1_LOSS/tree/main

https://github.com/openai/DALL-E