convolutional neural nets
(1.3 hours to learn)
Summary
Convolutional neural networks are a kind of feed-forward neural net architecture geared towards visual processing. In each layer, there are several groups of units whose weights are repeated (or "shared") across all spatial locations. The forward pass and backpropagation updates can both be computed efficiently using convolution.
Context
This concept has the prerequisites:
- learning invariances in neural nets (Convolutional nets are a neural net architecture that enforces invariance to translation.)
Core resources (read/watch one of the following)
-Free-
→ Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
- Lecture "Convolutional nets for digit recognition"
- Lecture "Convolutional nets for object recognition"
→ Gradient-based learning applied to document recognition
Supplemental resources (the following are optional, but you may find them useful)
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Section 5.5.6, pages 267-269
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location:
Section 16.5.1, pages 564-568
See also
- Convolutional nets are one way to get invariances in a neural net representation. Other ways include:
- augmenting the training set with warped examples
- tangent propagation , which penalizes instability with respect to transformations
- Tikhonov regularization , which penalizes instability with respect to noise
- Convolutional neural nets learn oriented filters similar to the receptive fields in the human primary visual cortex .