It has been a while since I last maintained this page. Use at your own peril.

Here is a machine learning to statistics translator I try to maintain. Most concepts are not really identical, so “translations” should be considered approximate. Based on the same idea in “All Of Statistics” by Larry Wasserman [p.15]. Many thanks to Ohad Shamir, and Saharon Rosset for helping populate the table. Despite my best efforts, it probably still contains errors. Drop me a note if you find any.

Concept Statistics Machine Learning
assumed model parameter space hypothesis class
  model hypothesis
  misspecified model agnostic learning
  deterministic outcome realizable learning
  sampling distribution generative model
output range improper learning
model types CART decision tree, axis parallel rectangles
  piecewise constant function decision list
  Bayes net directed acyclic graph (of conditional probabilities)
  latent variable model model based collaborative filtering
  neighborhood methods memory based collaborative filtering
  multivariate distribution graphical model
  Boolean circuit
  k-clause CNF
  k-term DNF
  Boolean formula
  Boolean threshold function
  Boolean circuit
  threshold circuit
  acyclic finite automata
tasks / problem setup estimation learning
  classification supervised learning
  clustering unsupervised learning
  transductive learning
  frequentist inference
  semi supervised learning
  support estimation manifold learning
  fixed design conditional model, discriminative model
  random design generative model
  adaptive design of experiments active learning
  MANOVA, vector regression structured learning
  basis augmentation feature creation
  missing data imputation collaborative filtering
  statistical process control semi supervised novelty detection
data data, sample, observations examples, training sample, instances
  validation sample
  test sample
  covariates, design, \(X\)-matrix features, attributes
methods M-estimation empirical risk minimization
  moment matching
  quantile matching
  U-estimation, V-estimation generative unsupervised RKHS learning
  Fisher’s LDA (assuming independence) Gaussian naive Bayes
interval methods confidence intervals PAC learnable
  credible interval PAC Bayes learnable
  fiducial interval
  prediction interval
error decomposition misspecification error approximation error
  risk estimation error, expected prediction error, test Error
  optimization Error
  optimism test error-training Error
  RSS empirical risk, training error
  Jackknife hypothesis stability
  model selection structural learning
problem complexity measures generalized degrees of freedom Rademacher complexity
  sample size sample complexity