RUS  ENG
Full version
JOURNALS // Sistemy i Sredstva Informatiki [Systems and Means of Informatics] // Archive

Sistemy i Sredstva Inform., 2022 Volume 32, Issue 4, Pages 14–20 (Mi ssi852)

Detection of distribution drift

A. A. Grushoa, N. A. Grushoa, M. I. Zabezhailoa, D. V. Smirnovb, E. E. Timoninaa, S. Ya. Shorgina

a Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119133, Russian Federation
b Sberbank of Russia, 19 Vavilov Str., Moscow 117999, Russian Federation

Abstract: Changing the properties of the data being collected is often referred to as data drift (various options for shifting the characteristics of the data). The existence of drift in artificial intelligence system training data often leads to a decrease in the efficiency of machine learning and erroneous solutions of artificial intelligence systems built on these data. In this regard, the problems of detecting drift in machine learning data, the moment of drift formation, and the consequences of changes in training data become relevant. The work proposes a method for detecting the drift of a probability distribution in an arbitrary metric space of large dimension. The method relies on the difference between unknown probability distributions in different regions of the original space in the event of drift. A drift model consisting of two different probability distributions is considered. Using the balls in metric space as the basis of the method allows one to create an efficient algorithm for calculating the ownership of data points to one of the balls associated with different distributions of the drift model. This circumstance seems to be essential for revealing the drift of a distribution in a high-dimensional space.

Keywords: distribution drift, mathematical statistics, efficiently calculated algorithm.

Received: 14.09.2022

DOI: 10.14357/08696527220402



© Steklov Math. Inst. of RAS, 2024