Publication:
Enhanced robustness of convolutional networks with a push-pull inhibition layer

Loading...
Thumbnail Image

Date

2020-01-23

Authors

Strisciuglio, Nicola
Lopez-Antequera, Manuel
Petkov, Nicolai

Advisors

Journal Title

Journal ISSN

Volume Title

Publisher

Springer london ltd
Metrics
Google Scholar
Export

Research Projects

Organizational Units

Journal Issue

Abstract

Convolutional neural networks (CNNs) lack robustness to test image corruptions that are not seen during training. In this paper, we propose a new layer for CNNs that increases their robustness to several types of corruptions of the input images. We call it a 'push-pull' layer and compute its response as the combination of two half-wave rectified convolutions, with kernels of different size and opposite polarity. Its implementation is based on a biologically motivated model of certain neurons in the visual system that exhibit response suppression, known as push-pull inhibition. We validate our method by replacing the first convolutional layer of the LeNet, ResNet and DenseNet architectures with our push-pull layer. We train the networks on original training images from the MNIST and CIFAR data sets and test them on images with several corruptions, of different types and severities, that are unseen by the training process. We experiment with various configurations of the ResNet and DenseNet models on a benchmark test set with typical image corruptions constructed on the CIFAR test images. We demonstrate that our push-pull layer contributes to a considerable improvement in robustness of classification of corrupted images, while maintaining state-of-the-art performance on the original image classification task. We released the code and trained models at the url http://github.com/nicstrisc/Push-Pull-CNN-layer.

Description

MeSH Terms

Benchmarking
Neural Networks, Computer
Inhibition, Psychological
Algorithms

DeCS Terms

Benchmarking
Mentoring
Methods
Neurons
Tidal Waves

CIE Terms

Keywords

Convolutional neural networks, Image corruption, Network robustness, Neuron response inhibition, push-pull layer, Orientation selectivity, Receptive-fields, Simple cell, Suppression, Broad, Model

Citation

Strisciuglio N, Lopez-Antequera M, Petkov N. Enhanced robustness of convolutional networks with a push–pull inhibition layer. Neural Comput Appl. 2020;32:17957–17971