Optimization methods for generalized tensor contraction

Tensor contraction is one of major operations defined in tensor calculus, a separate branch of mathematics, which become a fundamental language of theory of relativity, mechanics, electrodynamics, and solid state physics. Effective implementation of tensor contraction is of considerable practical significance for such areas as solving of mathematical physics problems, machine learning, spectral element methods, quantum chemistry, data mining, and high performance computing. In the last twenty years, the number of optimization methods for tensor contraction has increased and continues to grow. In this article, the author reviews widespread approaches for optimization of tensor contraction, which are used on single processor as well as multiprocessor systems with distributed memory. The review contains the description of methods for optimization of matrix and matrix-vector multiplications, important particular cases of tensor contraction, which are used as a base for the most tensor contraction optimizations. The described optimizations can be applied during program compilation performed by production compilers. The information provided in this work could be useful for systematizing knowledge.