The normal distribution is also known as the Gaussian distribution and is one of the most widely used continuous probability distributions. It is commonly employed to approximate the distributions of various data in natural and social phenomena. For example, measurement errors, human heights, and test scores are often said to follow a normal distribution.
The shape of a normal distribution is completely determined by two parameters: the mean ($\mu$) and the variance ($\sigma^2$).
Standard Normal Distribution
The most basic form of the normal distribution has a mean of 0 and a variance of 1, and is called the standard normal distribution. Its probability density function is:
$$ p(x) = \frac{1}{\sqrt{2\pi}} \exp\lbrace-\frac{x^2}{2}\rbrace $$
This function is normalized so that its integral over the entire real line equals 1.
Translation (Introducing the Mean $\mu$)
The standard normal distribution is centered at 0. To shift the center to an arbitrary $\mu$, we substitute $x - \mu$ for $x$.
$$ p(x | \mu) = \frac{1}{\sqrt{2\pi}} \exp\lbrace-\frac{(x - \mu)^2}{2}\rbrace $$
This is a normal distribution with mean $\mu$ and variance 1.
Scaling (Introducing the Variance $\sigma^2$)
To adjust the spread (variability) of the distribution, we introduce the variance $\sigma^2$. Taking the second derivative of the standard normal distribution’s PDF reveals that the inflection points are at $x = \pm 1$. To place these inflection points at $\sigma$, we substitute $x/\sigma$ for $x$ and adjust the normalization constant.
$$ p(x | 0, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\lbrace-\frac{x^2}{2\sigma^2}\rbrace $$
This is a normal distribution with mean 0 and variance $\sigma^2$.
General Normal Distribution
Combining the above translation and scaling yields the general normal distribution $\mathcal{N}(x | \mu, \sigma^2)$ with arbitrary mean $\mu$ and variance $\sigma^2$.
$$ \mathcal{N}(x | \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\lbrace-\frac{(x - \mu)^2}{2\sigma^2}\rbrace $$
References
- Taro Tezuka, “Understanding Bayesian Statistics and Machine Learning,” Kodansha (2017)