Publication: Enhanced robustness of convolutional networks with a push-pull inhibition layer
Loading...
Identifiers
Date
2020-01-23
Authors
Strisciuglio, Nicola
Lopez-Antequera, Manuel
Petkov, Nicolai
Advisors
Journal Title
Journal ISSN
Volume Title
Publisher
Springer london ltd
Abstract
Convolutional neural networks (CNNs) lack robustness to test image corruptions that are not seen during training. In this paper, we propose a new layer for CNNs that increases their robustness to several types of corruptions of the input images. We call it a 'push-pull' layer and compute its response as the combination of two half-wave rectified convolutions, with kernels of different size and opposite polarity. Its implementation is based on a biologically motivated model of certain neurons in the visual system that exhibit response suppression, known as push-pull inhibition. We validate our method by replacing the first convolutional layer of the LeNet, ResNet and DenseNet architectures with our push-pull layer. We train the networks on original training images from the MNIST and CIFAR data sets and test them on images with several corruptions, of different types and severities, that are unseen by the training process. We experiment with various configurations of the ResNet and DenseNet models on a benchmark test set with typical image corruptions constructed on the CIFAR test images. We demonstrate that our push-pull layer contributes to a considerable improvement in robustness of classification of corrupted images, while maintaining state-of-the-art performance on the original image classification task. We released the code and trained models at the url http://github.com/nicstrisc/Push-Pull-CNN-layer.
Description
MeSH Terms
Benchmarking
Neural Networks, Computer
Inhibition, Psychological
Algorithms
Neural Networks, Computer
Inhibition, Psychological
Algorithms
DeCS Terms
Benchmarking
Mentoring
Methods
Neurons
Tidal Waves
Mentoring
Methods
Neurons
Tidal Waves
CIE Terms
Keywords
Convolutional neural networks, Image corruption, Network robustness, Neuron response inhibition, push-pull layer, Orientation selectivity, Receptive-fields, Simple cell, Suppression, Broad, Model
Citation
Strisciuglio N, Lopez-Antequera M, Petkov N. Enhanced robustness of convolutional networks with a push–pull inhibition layer. Neural Comput Appl. 2020;32:17957–17971