Quality assessment of models and deep learning methods for super-resolution image formation

Authors

DOI:

https://doi.org/10.20535/SRIT.2308-8893.2025.4.06

Keywords:

single image super-resolution, quality assessment, generative models, deep learning methods, convolutional neural network, residual learning, recursive learning, fine-tuning of pre-trained models, perceptual metric, LPIPS, multicriteria decision analysis, DIV2K dataset, thresholds for practically acceptable and high-quality generated images

Abstract

This article examines evaluation metrics for the results of super-resolution image generation in solving the SISR task. The study comprises two experiments: the implementation of custom network architectures for SRGAN, VDSR, and SRCNN, and fine-tuning of pre-trained SRGAN, VDSR, and SRCNN models. An algorithm for assessing the quality of models and deep learning methods for generating super-resolution images is suggested. The VDSR model performed best in terms of pixel, structural, and perceptual metrics, as well as training time and visual confirmation by a human, highlighting that residual learning is more effective than recursive learning under the conditions of the two conducted experiments. Threshold values for practically acceptable and high-quality results were determined through visual analysis of many generated images and their corresponding quality metrics, including those reported by other researchers.

References

E. Agustsson, R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” 2017 IEEE Conference on Computer Vision and Pattern Recogni-tion Workshops (CVPRW), Honolulu, HI, USA, 21–26 July 2017. doi: https://doi.org/10.1109/cvprw.2017.150

Z. Wang, J. Chen, S.C.H. Hoi, “Deep Learning for Image Super-resolution: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 10, pp. 3365–3387, 2020. doi: https://doi.org/10.1109/tpami.2020.2982166

R. Timofte et al., “NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results,” 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 21–26 July 2017. doi: https://doi.org/10.1109/cvprw.2017.149

T. Ausare, “Ultimate Guide to Selecting a GPU for Deep Learning. Latest AI, ML & GPU Updates,” NeevCloud. Available: https://blog.neevcloud.com/ultimate-guide-to-selecting-a-gpu-for-deep-learning

F.A. Fardo, V.H. Conforto, F.C. de Oliveira, P.S. Rodrigues, A Formal Evaluation of PSNR as Quality Measurement Parameter for Image Segmentation Algorithms. 2016. doi: https://doi.org/10.48550/arXiv.1605.07116

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, Eero P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Im-age Processing, vol. 13, issue 4, pp. 600–612, 2004. doi: https://doi.org/10.1109/TIP.2003.819861

A. Mittal, A. Moorthy, A. Bovik, “Referenceless image spatial quality evaluation en-gine,” in 45th Asilomar Conference on Signals, Systems and Computers, vol. 38, pp. 53–54, 2011. doi: https://doi.org/10.1109/ACSSC.2011.6190099

A. Mittal, R. Soundararajan, A.C. Bovik, “Making a “completely blind” image quality analyser,” IEEE Signal Process. Lett., vol. 20, issue 3, pp. 209–212, 2013. doi: https://doi.org/10.1109/LSP.2012.2227726

N. Venkatanath, D. Praneeth, Bh. Maruthi Chandrasekhar, S.S. Channappayya, S.S. Medasani, “Blind image quality evaluation using perception based features,” 2015 Twenty First National Conference on Communications (NCC), Mumbai, India, 2015, pp. 1–6. doi: https://doi.org/10.1109/NCC.2015.7084843

R. Zhang, P. Isola, A.A. Efros, E. Shechtman, O. Wang, “The Unreasonable Effective-ness of Deep Features as a Perceptual Metric,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 586–595. doi: https://doi.org/10.1109/CVPR.2018.00068

N.I. Nedashkovskaya, “Method for weights calculation based on interval multiplicative pairwise comparison matrix in decision-making models,” Radio Electronics, Computer Science, Control, no. 3, pp. 155–167, 2022. doi: https://doi.org/10.15588/1607-3274-2022-3-15

N.I. Nedashkovskaya, “Estimation of the accuracy of methods for calculating interval weight vectors based on interval multiplicative preference relations,” IEEE 3rd Interna-tional Conference on System Analysis & Intelligent Computing (SAIC), 2022. doi: https://doi.org/10.1109/SAIC57818.2022.9922977

N.I. Nedashkovskaya, “Method for Evaluation of the Uncertainty of the Paired Compari-sons Expert Judgements when Calculating the Decision Alternatives Weights,” Journal of Automation and Information Sciences, vol. 47, issue 10, pp. 69–82, 2015. doi: https://doi.org/10.1615/JAutomatInfScien.v47.i10.70

N.D. Pankratova, N.I. Nedashkovskaya, “Hybrid Method of Multicriteria Evaluation of Decision Alternatives,” Cybernetics and Systems Analysis, vol. 50, no. 5, pp. 701–711, 2014. doi: https://doi.org/10.1007/s10559-014-9660-2

N.I. Nedashkovskaya, “Investigation of methods for improving consistency of a pairwise comparison matrix,” Journal of the Operational Research Society, vol. 69, no. 12, pp. 1947–1956, 2018. doi: https://doi.org/10.1080/01605682.2017.1415640

C. Ledig et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Ad-versarial Network,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 21–26 July 2017, pp. 105–114. doi: https://doi.org/10.1109/cvpr.2017.19

J. Kim, J.K. Lee, K.M. Lee, “Accurate Image Super-Resolution Using Very Deep Con-volutional Networks,” 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion (CVPR), Las Vegas, NV, USA, 27–30 June 2016, pp. 1646–1654. doi: https://doi.org/10.1109/cvpr.2016.182

J. Kim, J.K. Lee, K.M. Lee, “Deeply-Recursive Convolutional Network for Image Su-per-Resolution,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016, pp. 1637–1645, 2016. doi: https://doi.org/10.1109/cvpr.2016.181

C. Dong et al., “Image Super-Resolution Using Deep Convolutional Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295–307, 2016. doi: https://doi.org/10.1109/tpami.2015.243928

A.A. Lanko, N.I. Nedashkovskaya, “Generative models and methods of deep learning for the SISR problem,” System sciences and informatics: collection of reports of the 3rd All-Ukrainian scientific and practical conference “System sciences and informatics”, November 25–29, 2024, Kyiv. K.: IASA KPI, 2024, pp. 176–181. Available: http://mmsa.kpi.ua/sites/default/files/systemni_nauky_ta_informatyka_2024.pdf

K. He et al., “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification,” 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015, pp. 1026–1034. doi: https://doi.org/10.1109/iccv.2015.123

X. Glorot, Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” Proceedings of the Thirteenth International Conference on Artificial Intelli-gence and Statistics (AISTATS), Sardinia, Italy, 13–15 May 2010, PMLR, vol. 9, pp. 249–256. Available: http://proceedings.mlr.press/v9/glorot10a.html

A. Lugmayr et al., “SRFlow: Learning the Super-Resolution Space with Normalizing Flow,” Computer Vision – ECCV 2020, Cham, 2020, pp. 715–732. doi: https://doi.org/10.1007/978-3-030-58558-7_42

Q. Jiang et al., “Single Image Super-Resolution Quality Assessment: A Real-World Da-taset, Subjective Studies, and an Objective Metric,” IEEE Transactions on Image Pro-cessing, vol. 31, pp. 2279–2294, 2022. doi: https://doi.org/10.1109/tip.2022.3154588

N.I. Nedashkovskaya, “A system approach to decision support on basis of hierarchical and network models,” System Research and Information Technologies, no. 1, pp. 7–18, 2018. doi: https://doi.org/10.20535/srit.2308-8893.2018.1.01

A. Ignatov et al., “PIRM challenge on perceptual image enhancement on smartphones: report,” Conference on Computer Vision (ECCV) Workshops, 2019. doi: https://doi.org/10.1007/978-3-030-11021-5_20

Dandan Gao, Dengwen Zhou, “A very lightweight and efficient image super-resolution network,” Expert Systems with Applications, vol. 213, Part A, 1, March 2023, 118898. doi: https://doi.org/10.1016/j.eswa.2022.118898

“GitHub - tensorlayer/SRGAN: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” GitHub. Available: https://github.com/tensorlayer/SRGAN

“GitHub - twtygqyy/pytorch-vdsr: VDSR (CVPR2016) pytorch implementation,” GitHub. Available: https://github.com/twtygqyy/pytorch-vdsr.

“GitHub - Lornatang/SRCNN-PyTorch: Pytorch framework can easily implement srcnn algorithm with excellent performance,” GitHub. Available: https://github.com/Lornatang/SRCNN-PyTorch

B. Lim, S. Son, H. Kim, S. Nah, K.M. Lee, “Enhanced deep residual networks for sin-gle image super-resolution,” IEEE Conference on Computer Vision and Pattern Recogni-tion Workshops (CVPRW), 2017, pp. 1132–1140. doi: https://doi.org/10.1109/CVPRW.2017.151

X. Wang et al., “ESRGAN: Enhanced super-resolution generative adversarial networks,” Computer Vision – ECCV 2018 Workshops: Munich, Germany, September 8-14, 2018, Proceedings, Part V, pp. 63–79. doi: https://doi.org/10.1007/978-3-030-11021-5_5

Downloads

Published

2025-12-29

Issue

No. 4 (2025)

Section

Methods, models, and technologies of artificial intelligence in system analysis and control

License

This is an open access journal which means that all content is freely available without charge to the user or his/her institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open access.