Overview of visualization methods for artificial neural networks

Modern algorithms based on artificial neural networks (ANNs) are extremely useful in solving a variety of complicated problems in computer vision, robust control, and natural language analysis of sound and texts as applied to data processing, robotics, etc. However, for the ANN approach to be successfully incorporated into critically important systems, for example, in medicine or jurisprudence, a clear interpretation is required for the internal architecture of ANN and for ANN-based decision-making processes. In recent years, analysis methods based on various visualization techniques applied to computation graphs, loss function profiles, parameters of single network layers, and even single neurons have become especially popular as tools for creating explainable deep learning models. This survey systematizes existing mathematical analysis methods and explanations of the behavior of underlying algorithms and presents formulations of corresponding problems in computational mathematics. The study and visualization of deep neural networks are new poorly studied yet rapidly developing areas. The considered methods give a deeper insight into the operation of neural network algorithms.