Introduction
In one of the previous notes, we discussed several common families of distributions. In this note, we discuss the three types of parameters that help in constructing families of distributions. The resulting families have ready physical interpretations that make them useful for modeling, as well as convenient mathematical properties.
The three types of parameters are location, scale (or rate), and shape. These three can also be understood in terms of elementary transformations in mathematics and their behaviours.
1. Location parameter
A location parameter corresponds to an additive transformation, producing a translation of the distribution along the x-axis, without changing its shape or spread. If $X$ has PDF $f(x)$, then $Y = X + \mu$ has PDF $f(x-\mu)$, where $\mu$ is the location parameter.
2. Scale (or rate) parameter
A scale parameter corresponds to a multiplicative transformation, producing an expansion or contraction of the distribution. If $X$ has PDF $f(x)$, then $Y = \sigma X$ has PDF $\frac{1}{\sigma}f(x/\sigma)$, where $\sigma$ is the scale parameter. A rate parameter is simply the reciprocal of a scale parameter, $\lambda = 1/\sigma$, and is common in waiting-time models.
3. Shape parameter
A shape parameter corresponds to a nonlinear (often exponential or power) transformation that alters the fundamental shape of the distribution, such as its skewness, number of modes, or tail behaviour, rather than merely shifting or stretching it.
These ideas will be illustrated further with examples in the sections that follow.
Poisson distribution
$$f_X(x|\lambda) = \begin{cases} \dfrac{\lambda^x e^{-\lambda}}{x!} & x = 0,1,2,\dots \\ 0 & \text{otherwise} \end{cases}$$
$\lambda > 0$ is the rate parameter.
Uniform distribution
$$f_X(x|\mu,\theta) = \begin{cases} \dfrac{1}{\theta} & \mu < x < \mu+\theta \\ 0 & \text{elsewhere} \end{cases}$$
$\mu$ is the location parameter (lower limit), $\theta > 0$ is the scale parameter (range length).
Beta distribution
$$f_X(x|\alpha,\beta) = \begin{cases} \dfrac{1}{B(\alpha,\beta)}\,x^{\alpha-1}(1-x)^{\beta-1} & 0 < x < 1 \\ 0 & \text{elsewhere} \end{cases}$$
$\alpha > 0$ and $\beta > 0$ are both shape parameters.
Gamma distribution
$$f_X(x|\alpha,\beta) = \begin{cases} \dfrac{1}{\Gamma(\alpha)\,\beta^{\alpha}}\,x^{\alpha-1}e^{-x/\beta} & x > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$\alpha > 0$ is the shape parameter, $\beta > 0$ is the scale parameter (equivalently $\tau = 1/\beta$ is the rate parameter).
Exponential distribution
$$f_X(x|\tau) = \begin{cases} \tau e^{-\tau x} & x > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$\tau > 0$ is the rate parameter (equivalently $\lambda = 1/\tau$ is the scale parameter).
Weibull distribution
$$f_X(x|\alpha,\theta) = \begin{cases} \dfrac{\alpha}{\theta}\left(\dfrac{x}{\theta}\right)^{\alpha-1} e^{-(x/\theta)^{\alpha}} & x > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$\alpha > 0$ is the shape parameter, $\theta > 0$ is the scale parameter.
Normal distribution
$$f_X(x|\mu,\sigma) = \dfrac{1}{\sqrt{2\pi}\sigma}\,e^{-(x-\mu)^2/(2\sigma^2)}, \quad -\infty < x < \infty$$
$\mu$ is the location parameter, $\sigma > 0$ is the scale parameter (variance), equivalently $\tau = 1/\sigma$ is the rate parameter.
In certain cases, the parameters are representing other characteristics of the distribution
Bernoulli distribution
Number of success (x)
$$f_X(x|\theta) = \begin{cases} \theta^x(1-\theta)^{1-x} & x = 0, 1 \\ 0 & \text{elsewhere} \end{cases} $$
$\theta \in (0,1)$ is the success-probability parameter.
Binomial Distribution
Number of successes (x)
$$f_X(x|n,\theta) = \begin{cases} \displaystyle\binom{n}{x} \theta^x (1-\theta)^{n-x} & x = 0, 1, 2, \cdots n \\ 0 & \text{elsewhere} \end{cases} $$
Here $n$ and $\theta$ are parameters. $\theta \in (0,1)$ is the success-probability parameter.
Negative Binomial distribution
fixed number of successes $r$
$$f_X(x|r,\theta) = \begin{cases} \dbinom{x+r-1}{x}(1-\theta)^{x}\theta^{r} & x = 0,1,2,\dots \\ 0 & \text{otherwise} \end{cases}$$
$r$ is fixed (known number of successes), $\theta \in (0,1)$ is the success-probability parameter.
Geometric distribution
$$f_X(x|\theta) = \begin{cases} (1-\theta)^{x}\theta & x = 0,1,2,\dots \\ 0 & \text{otherwise} \end{cases}$$
$\theta \in (0,1)$ is the success-probability parameter.