J. D. Agafonova, A. V. Gaidel, P. M. Zelter, A. V. Kapishnikov, A. V. Kuznetsov, E. N. Surovtsev, A. V. Nikonorov, “Joint analysis of radiological reports and CT images for automatic validation of pathological brain conditions”, Computer Optics, 2023, Volume 47, Issue 1,Pages <nobr>152

IMAGE PROCESSING, PATTERN RECOGNITION

Joint analysis of radiological reports and CT images for automatic validation of pathological brain conditions

J. D. Agafonova^a, A. V. Gaidel^ab, P. M. Zelter^c, A. V. Kapishnikov^c, A. V. Kuznetsov^ade, E. N. Surovtsev^c, A. V. Nikonorov^ab

^a Samara National Research University
^b Image Processing Systems Institute of the RAS - Branch of the FSRC "Crystallography and Photonics" RAS, Samara, Russia, Samara
^c Samara State Medical University
^d Artificial Intelligence Research Institute, Moscow
^e Sber AI, 121170, Moscow, Russia, Kutuzovsky prospekt, 32 building 2

Abstract: We consider a problem of validation of radiological medical reports and computed tomography images for an automated analysis of brain structures. Two methods for solving the problem are proposed: a method based on the ruCLIP multimodal model, and a method based on the joint use of two separate classifiers – for a text report and for a brain CT image. We discuss methods evaluation and the obtained results. The proposed approaches make it possible to correctly classify 99.6

Keywords: deep learning, computed tomography, computer-aided diagnosis, pattern recognition, natural language processing

Received: 01.08.2022
Accepted: 10.10.2022

DOI: 10.18287/2412-6179-CO-1201