Abstract:
We discuss issues of modular learning in artificial neural networks and explore possibilities of
the partial use of modules when the computational resources are limited. The proposed method is
based on the ability of a wavelet transform to separate information into high- and low-frequency
parts. Using the expertise gained in developing convolutional wavelet neural networks, the authors
perform a transverse-layer partitioning of the network into modules for the further partial use on
devices with low computational capability. The theoretical justification of this approach in the paper is supported by experimentally dividing the MNIST database into 2 and 4 modules before using them sequentially and measuring the respective accuracy and performance. When using the individual modules, a two-fold (or higher) performance gain is achieved. The theoretical statements
are verified using an AlexNet-like network on the GTSRB dataset, with a performance gain of
33% per module with no loss of accuracy.