PyTorch implementation of "Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment" (arXiv/AAAI), which has been accepted by AAAI-2024.

This code is modified from FAST-VQA.

For image quality assessment (IQA), please refer to IQA/demo_train_iqa_baseline.py.

For video quality assessment (VQA), please refer to VQA/demo_train.py to get the training result, and refer to VQA/demo_finetune.py to get the finetuning result. We also provide the training log for VQA.

The main idea/contribution lies in the data sampling, which can be found in IQA and VQA.

Make sure the configuration has been properly set in

• fast-sama-iqa.yml for IQA training;
• fast-SAMA-train.yml for VQA training on LSVQ;
• and fast-SAMA-tinetune.yml for VQA finetuning.

And please prepare the pretrained models of video-swin for VQA and swin-v2 for IQA.

We have provided the pretrained weights (trained on LSVQ train set): GoogleDrive / BaiDu (Code:xyns). please check the pretrained weights in ./VQA/pretrained_weights folder and put the weights in the folder.

To test on your own dataset or video files, please construct the dataset information as the examplar in ./VQA/examplar_data_labels, and set the configuration in fast-SAMA-test.yml. Run the file demo_test.py to check the details.

Different environment may induce possible fluctuation of performance.

Python 3.8.10
PyTorch 1.7.0

The installation can refer to FAST-VQA.

If you are interested in the work, or find the code helpful, please cite our work

@article{sama2024,
        title={Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment},
        volume={38},
        number={4},
        journal={Proceedings of the AAAI Conference on Artificial Intelligence},
        author={Liu, Yongxu and Quan, Yinghui and Xiao, Guoyao and Li, Aobo and Wu, Jinjian},
        year={2024},
        month={Mar.},
        pages={3792-3801},
        url={https://ojs.aaai.org/index.php/AAAI/article/view/28170},
        DOI={10.1609/aaai.v38i4.28170}
}

Feel free to contact me via [email protected] if any question or bug.