Iterative convex estimation of linear regression models under data stochastic heterogeneity

One of the key challenges in linear regression analysis is ensuring robust parameter estimation under stochastic data heterogeneity. In such cases, classical least squares estimates lose their stability. This problem becomes particularly acute with error distributions having heavier tails than normal distribution. Among various approaches to enhance regression robustness, replacing quadratic loss functions with convex-concave ones has been considered, though direct application leads to multimodal objective functions, significantly complicating the optimization problem.
This study aims to analyze properties of variationally-weighted quadratic and absolute approximations for non-convex loss functions. We propose an approach based on replacing the original non-convex regression problem with iterative application of weighted least squares and least absolute deviations methods, effectively implementing variationally-weighted approximations for non-convex loss functions. Each iteration of the weighted least absolute deviations method employed descent algorithms along nodal lines.
Through Monte Carlo simulations with various loss functions, we demonstrate that the weighted least absolute deviations method outperforms least squares in computational efficiency while maintaining comparable estimation accuracy. When multiple regression assumptions are violated simultaneously, either the weighted least absolute deviations method or the generalized least absolute deviations method (implemented as a generalized descent algorithm) proves preferable for achieving acceptable accuracy. We provide computational complexity estimates and execution time analyses depending on sample size and number of regression parameters.