The accuracy-stability paradox and the boundaries of verifiable accuracy, robustness, and generalisation in deep learning

In this lecture, we will delve into the theoretical limitations of determining the guaranteed stability and accuracy of neural networks in classification tasks. We will consider classical distribution-agnostic framework and algorithms minimising empirical risks and potentially subjected to some weights regularisation. We will show that there is a large family of tasks for which computing and verifying ideal stable and accurate neural networks in the above settings is extremely challenging, if at all possible, even when such ideal solutions exist within the given class of neural architectures.