Simple method to improve quality of sparse models training from scratch

Existing pruning algorithms can achieve good quality on sparse neural networks. But the recieved sparse architectures, when training from scratch, often can't achive the same quality as pruning (especially for very sparse networks).
In this work the weights restoring method to improve training from scratch quality is described.