Gradient mask: lateral inhibition mechanism improves performance in artificial neural networks

In this paper we propose Gradient Mask, which helps the network to filtering out noisy or unimportant features while training. We propose a new criterion for gradient quality which can be used as a measure during training of various convolutional neural networks (CNNs). We demonstrate analytically how lateral inhibition in artificial neural networks improves the quality of propagated gradients. Finally, we conduct several different experiments to study how Gradient Mask improves the performance of the network both quantitatively and qualitatively.