On problems of measuring the quality of neural network algorithms for video processing

The benchmarks of the MSU laboratory are in the top of the tasks "Image Quality Assessment" (1st and 2nd place out of 3), "Video Quality Assessment" (1st, 4th and 6th place out of 10) and "Video Super-Resolution" (1st, 2nd, 3rd place out of 16); in other words, the most measured open-source methods in their own category for different subtasks. There have also been 3 video quality challenges at ECCV 2024, a challenge at CVPR 2025, and 3 challenges are planned for ICCV 2025. Over the past 4 years, 5 publications have been made at A* conferences (ICML, AAAI, ICLR and NeurIPS), mainly devoted to the sustainability of image and video quality metrics. If it were possible to construct a stable metric, it could be used as a loss function. However, this problem remains unresolved, as well as several other problems that have clearly manifested themselves in benchmarks and challenges. Serious progress in the field and its new big challenges will be discussed.