We found 25 relevant concepts
  1. structured mean field

    The naive mean field approximation assumes a fully factorized approximating distribution, which can be inaccurate if variables are tightly coupled. Structured mean field instead assumes the distribution factorizes into a product of tractable distributions, such as trees or chains.
  2. comparing Gaussian mixtures and k-means

    Gaussian mixture models and K-means are two canonical approaches to clustering, i.e. dividing data points into meaningful groups. This concept node discusses the tradeoffs between them.
  3. mean field approximation

    In variational inference algorithms, we try to approximate an intractable distribution with a tractable one. Mean field is probably the most common example. The approximating distribution is factorized into independent terms corresponding to different variables or groups of variables. Variational Bayes and variational Bayes EM are important applications of mean field to Bayesian parameter estimation.
  4. k-means++

    The k-means++ algorithm is a simple stochastic procedure for choosing the initial cluster centers for the classic k-means algorithm. This initialization guarantees that the expected objective of the final clustering solution will be within a constant factor of the optimal objective. Briefly stated, k-means++ selects the initial cluster centers as follows: the first center is chosen randomly from the input data points. Next, each subsequent cluster center is chosen from the remaining data points with probability proportional to its squared distance from the point's closest existing cluster center.
  5. Student-t distribution

    The student-t distribution is a continuous probability distribution motivated by estimating the mean of a Gaussian population with unknown variance.
  6. F measure

    The F measure (F1 score or F score) is a measure of a test's accuracy and is defined as the weighted harmonic mean of the precision and recall of the test.
  7. Gaussian BP on trees

    Marginalization in Gaussian MRFs can be performed in cubic time by inverting a matrix, but this is too slow for some applications. If the model is tree-structured, belief propagation can compute the means and single-node variances in linear time. Unlike in general MRFs, it turns out that the loopy version yields the correct means.
  8. k-means

    K-means is a clustering algorithm, i.e. a way of partitioning a set of data points into "clusters," or sets of data points which are similar to one another. It works by iteratively reassigning data points to clusters and computing cluster centers based on the average of the point locations. It is commonly used for vector quantization and as an initialization for Gaussian mixture models.
  9. variational inference and exponential families

    Variational inference algorithms based on mean field turn out to have especially nice forms in exponential family models with appropriate conjugate structure.
  10. multivariate Gaussian distribution

    The multivariate Gaussian distribution is a generalization of the Gaussian distribution to higher dimensions. The parameters of an n-dimension multivariate Gaussian distribution are an n-dimensional mean vector and an n-by-n dimensional covariance matrix.
  11. probit regression

    Probit regression is a discriminative model for classification. In this model, the binary targets are generated by sampling latent Gaussian variables whose means are linear in the inputs, and passing them through a threshold.
  12. comparing normal populations

    A common task in statistics is to determine whether two normally distributed populations have the same mean. The appropriate test can depend on factors such as the sample size and whether the populations are paired or independent.
  13. variational Bayes

    Bayesian parameter estimation often results in an intractable posterior over model parameters. Variational Bayes is an application of variational inference (in particular, mean field) to approximating the marginals over parameters as well as the marginal likelihood.
  14. bases

    A basis is a set of linearly independent vectors that define a coordinate system for a vector space, where the basis vectors span the entire vector space (meaning that any vector in this space can be represented as a linear combination of the basis vectors).
  15. computations on multivariate Gaussians

    Multivariate Gaussians are widely used in computational sciences because many useful operations can be performed efficiently. Marginalization is easy: we simply pull the relevant rows and columns of the mean and covariance. Conditioning can be done with a matrix inversion.
  16. Dirichlet process

    The Dirichlet process is a stochastic process that defines a probability distribution over infinite-dimensional discrete distributions, meaning that a draw form a DP is itself a distribution (with a countably infinite number of parameters). Its name stems from the fact that the marginal of a DP for any finite partition is Dirichlet distributed. While the DP is often discussed alongside the Chinese Restaurant Process (CRP), the two are not the same entity. The DP is the de Finetti mixing measure for the CRP, meaning that sampling i.i.d. from a draw of a DP is equivalent to sequentially drawing samples from the CRP.
  17. Bayesian parameter estimation: Gaussian distribution

    Using the Bayesian framework, we can infer the mean parameter of a Gaussian distribution, the scale parameter, or both. Since Gaussians are widely used in probabilistic modeling, the computations that go into this are common motifs in Bayesian machine learning more generally.
  18. linear-Gaussian models

    A linear-Gaussian model is a Bayes net where all the variables are Gaussian, and each variable's mean is linear in the values of its parents. They are widely used because they support efficient inference. Linear dynamical systems are an important special case.
  19. information form for multivariate Gaussians

    While we normally represent multivariate Gaussians in terms of their mean and covariance, information form is often a useful alternative. The distribution is represented in terms of a quadratic "energy function." This representation is convenient for conditioning, and is the basis for Gaussian Markov random fields.
  20. Bayesian parameter estimation: multivariate Gaussians

    Using the Bayesian framework, we can infer the posterior over the mean vector of a multivariate Gaussian, the covariance matrix, or both. Since multivariate Gaussians are widely used in probabilistic modeling, the computations that go into this are common motifs in Bayesian machine learning more generally.