ImageNet Classification with Deep Convolutional Neural Networks
📜 Abstract
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
✨ Summary
Summary
The paper presents a large supervised convolutional neural network, later widely known as AlexNet, trained on ImageNet for 1000-class image classification. The model used five convolutional layers followed by three fully connected layers, ReLU activations, max-pooling, local response normalization, dropout, data augmentation, stochastic gradient descent with momentum, and training distributed across two GPUs. It contained approximately 60 million parameters and was trained on roughly 1.2 million images.
The central empirical result was a top-1 error rate of 37.5% and a top-5 error rate of 17.0% on ILSVRC-2010, substantially improving on previous methods. A related ensemble and pretraining strategy achieved a 15.3% top-5 error rate in ILSVRC-2012, compared with 26.2% for the second-best competition entry. The experiments also showed that ReLU activations substantially accelerated optimization, data augmentation and dropout reduced overfitting, and increasing convolutional depth improved accuracy.
The paper had a documented influence on subsequent computer-vision research. VGGNet explicitly describes its architecture as an effort to improve the original Krizhevsky et al. configuration, retains design elements such as ReLU activations, 224×224 inputs, three fully connected layers, and ImageNet training, and replaces the large early filters with deeper stacks of 3×3 convolutions. VGGNet models were subsequently used as transferable feature representations for object detection, semantic segmentation, image captioning, and other recognition tasks. (arxiv.org)
NeurIPS later identified the work as a paper with a major impact on the machine-learning community, noting that its 2012 ImageNet result far surpassed the state of the art at the time. (blog.neurips.cc) The paper therefore contributed concrete architectural and training practices that became standard reference points for later deep visual-recognition systems, while its strongest directly documented influence is in subsequent ImageNet-based CNN research. (papers.nips.cc)