Notes on Statistics and Machine Learning

Notes on Statistics and Machine Learning

A small encyclopedia for looking up a concept, understanding it, and moving on to the next one. Each article starts with the intuition, then gives the mathematics, a worked example, and links to related ideas. Browse the contents below or use the alphabetical index.

Statistics and Probability

Statistical Foundations

What statistics is, where data come from, and how to summarize a dataset.

  1. Introduction to Statistics
  2. Populations and Samples
  3. Types of Data and Variables
  4. Descriptive Statistics
  5. Mean, Median, and Mode
  6. Variance and Standard Deviation
  7. Quantiles, Percentiles, and Outliers

Probability

The rules of chance: events, conditioning, independence, and random variables.

  1. Introduction to Probability
  2. Conditional Probability
  3. Statistical Independence
  4. Bayes' Theorem
  5. Random Variables
  6. Expected Value
  7. Variance of a Random Variable

Probability Distributions

The standard models for random quantities and how their parameters shape them.

  1. Probability Distributions
  2. Bernoulli and Binomial Distributions
  3. Normal Distribution
  4. Poisson Distribution
  5. Uniform Distribution
  6. Exponential Distribution

Statistical Inference

Learning about a population from a sample, and quantifying the uncertainty.

  1. Sampling Distributions
  2. Law of Large Numbers
  3. Central Limit Theorem
  4. Point Estimation
  5. Likelihood and Maximum Likelihood Estimation
  6. Confidence Intervals
  7. Hypothesis Testing
  8. P-values and Statistical Significance

Relationships Between Variables

Measuring and modelling how two quantities vary together.

  1. Covariance
  2. Correlation
  3. Simple Linear Regression

Planned subjects: machine learning, deep learning, and the underlying mathematics (linear algebra, calculus, optimization).