sparse coding
(1.5 hours to learn)
Summary
Sparse coding is a probabilistic model of natural images where each region of an image is represented as a linaer combination of a small number of components drawn from a dictionary. When the model is fit to natural images, the dictionary elements resemble the receptive fields of cells in the primary visual cortex.
Context
This concept has the prerequisites:
- maximum likelihood
- matrix multiplication (Sparse coding is a kind of matrix factorization.)
- heavy-tailed distributions (Sparse coding uses heavy-tailed distributions for the coefficients.)
- optimization problems (Fitting sparse coding requires solving an optimization problem.)
Core resources (read/watch one of the following)
-Free-
→ Emergence of simple-cell receptive field properties by learning a sparse code for natural images
-Paid-
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location:
Section 13.8, pages 468-474
See also
-No Additional Notes-