Abstract:
In this paper, we investigate a method of approximate entropy estimation, designed to speed up the classical method of maximum entropy estimation due to the use of regularization in the optimization problem. This method is compared with the method of maximum likelihood and Bayesian estimation, both experimentally and in terms of theoretical calculations for some special cases. Estimation methods are tested on the example of a linear regression problem with errors of various types, including asymmetric distributions as well as a multimodal distribution in the form of a mixture of Gaussian components.
Keywords:probabilistic mathematical model, maximum entropy method, linear regression, regularization, errors distribution.