Introduction
Random variable characterises a random phenomenon by listing the range and the corresponding probability distribution (that is, pmf in the discrete case or pdf in the continuous case). For example we can find a pdf for the water consumption of a city for any given day. Here, we are interested in finding a most probable value for this random event (consumption of water); that is on “expected” consumption of water. The following notes deals with expectation $E(X)$ expectation of a random variable $X$ or more generally expectation $E[g(x)]$ of a random variable.
Expectation of a Random Variable
Definition
If $X$ is a discrete random variable and $f_X(x)$ is the value of its pmf at $x$, then the expected value of $X$ is
$$ E(X) = \sum_{x} x \cdot f_X(x)$$
provided the sum in RHS exists. Otherwise, the mathematical expectation is undefined.
Correspondingly, if $X$ is a continuous random variable and $f_X(x)$ is the value of its pdf at $x$, the expected value of $X$ is
$$E(X) = \int_{-\infty}^{\infty} x f_X(x) \, dx $$
provided the integral in RHS exists. Otherwise, the mathematical expectation is undefined.
Infact, the concept of a mathematical expectation defines a value, the random variable is expected to assume an “average” value, when the random experiment is repeated under the given conditions. A mathematical expectations arose in connection with the games of chance.
Expectation of functions of a Random Variable
Let $X$ be a random variable with probability distribution $f_X(x)$. There are many problems in which we are interested not only in the expected value of a random variable $X$ but also in the expected values of random variables related to $X$ That is we might be interested in the random variable $Y$. Whose values are related to the value of $X$ by means of a relation (function) $Y = g_X(x)$. Let us define the expectation of $g_X(x)$ as follows.
$$ E[g(X)] = \begin{cases} \displaystyle\sum_{x} g_X(x) f_X(x) & X \text{ is discrete RV} \\[10pt] \displaystyle\int_{-\infty}^{\infty} g_X(x) f_X(x) \, dx & X \text{ is continuous RV} \end{cases} $$
Some authors prove the above statement and hence adopt it in the numerical problems we assume the above statement as a formula to find $E[g(X)]$ and apply it in the subsequent discussions, and in numerical examples.
Example 1
If $X$ is the number, obtained when a balanced die is rolled, find the expected value of $Y = g(X) = X^2 + 2X + 1$.
Since $X$ is a discrete random variable with $A = \{1, 2, 3, 4, 5, 6\}$ and each of these possible outcome has the probability $\dfrac{1}{6}$, we have,
$$\begin{array}{c|cccccc} X = x & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline f_X(x) & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} \end{array}$$
$$E[g(X)] = \sum_{x} g_X(x) f_X(x) \quad \text{(According to the definition)}$$
$$= \sum_{x} (X^2 + 2X + 1) f_X(x)$$
$$= (1^2 + 2(1) + 1)f(1) + (2^2 + 2(2) + 1)f(2) + (3^2 + 2(3) + 1)f(3) +$$
$$(4^2 + 2(4) + 1)f(4) + (5^2 + 2(5) + 1)f(5) + (6^2 + 2(6) + 1)f(6) $$
$$= \frac{139}{6} $$
Moments of Univariate Distribution:
We define moments of a random variable in two ways of which the first definition is important in statistics because they serve to describe the shape of the distribution of a random variable.
Definition 1:
The $r^{th}$ moment above the mean $(\mu)$ of a random variable $X$, denoted by $\mu_r$ or $m_r$ is
$$\mu_r = \begin{cases} \displaystyle\sum_{x} (x-\mu)^r f_X(x) & \text{if } X \text{ is discrete rv} \\[10pt] \displaystyle\int_{-\infty}^{\infty} (x-\mu)^r f_X(x) \, dx & \text{if } X \text{ is continuous rv} \end{cases} $$
for $r = 0, 1, 2, 3, \cdots$
that is $\mu_r$ is defined as the expected value of $(X-\mu)^r$ or
$$ \mu_r = E[(X-\mu)^r] $$
Definition 2:
The $r^{th}$ moment about the origin of a random variable $X$, denoted by $\mu_r’$ or $m_r’$ is the expected value of $X^r$ that is $\mu_r’ = E(X^r)$.
$$\mu_r’ = \begin{cases} \displaystyle\sum_{x} x^r f_X(x) & X \text{ is discrete rv} \\[10pt] \displaystyle\int_{-\infty}^{\infty} x^r f_X(x) \, dx & X \text{ is continuous rv} \end{cases}$$
for $r = 0, 1, 2, 3 \cdots$
We can observe from the definitions that
$$\mu_0 = E[(X-\mu)^0] = E(1) = 1 \quad \text{and} \quad \mu_0′ = E[X^0] = E(1) = 1$$
for any random variable $X$.
Also $\mu_1′ = E(X)$ which is the expected value of the random variable $X$ which we define as the mean of the distribution of $X$ or simply mean of $X$ and it is denoted as $\mu$ or $\mu_X$. Hence,
$$\mu_1 = E(X – \mu)$$
$$= E(X) – E(\mu)$$
$$= \mu – \mu \quad (\because \mu \text{ is a constant})$$
$$\mu_1 = 0$$
We shall derive an expression which calculates $\mu_r$ in terms of $\mu_r’$ and define some special moments $\mu_2, \mu_3$ and $\mu_4$ which are of special importance in statistics.
Relationship between $\mu_r$ and $\mu_r’$
Let us derive from $r = 2$ since we had already discussed for $r = 1$.
Let $$r = 2$$
$$\mu_2 = E[(X-\mu)^2]$$
$$= E[X^2 + \mu^2 – 2\mu X]$$
$$= E(X^2) + E(\mu^2) – 2\mu E(X)$$
$$= E(X^2) + \mu^2 – 2\mu \cdot \mu$$
$$= E[X^2] – \mu^2$$
$$= \mu_2′ – (\mu_1′)^2 \quad \text{or} \quad E(X^2) – E(X)^2$$
$$\therefore \quad \mu_2 = E(X^2) – E(X^2) \quad \text{or equivalently}$$
$$\mu_2 = \mu_2′ – (\mu_1′)^2 = \mu_2′ – \mu^2$$
Let $r = 3$
$$\mu_3 = E[(X-\mu)^3]$$
$$= E[X^2 – 3X^2\mu + 3X\mu^2 – \mu^3]$$
$$= E(X^3) – 3\mu E(X^2) + 3\mu^2 E(X) – \mu^3$$
$$= E(X^3) – 3\mu E(X^2) + 3\mu^2\mu – \mu^3$$
$$\mu_3 = E(X^3) – 3\mu E(X^2) + 2\mu^3 $$
$$\mu_3 = E(X^3) – 3E(X)E(X^2) + 2E(X)^3 \quad \text{or equivalently}$$
$$\mu_3 = \mu_3′ – 3\mu_1’\mu_2′ + 2(\mu_1′)^3$$
Similarly, we can derive an expression for $\mu_4$
let $r=4$
$$\mu_4 = E[(X-\mu)^4]$$
$$= E[X^4 – 4X^3\mu + 6X^2\mu^2 – 4X\mu^3 + \mu^4]$$
$$= E(X^4) – 4E(X^3)\mu + 6E(X^2)\mu^2 – 4E(X)\mu^3 + \mu^4$$
$$= E(X^4) – 4E(X^3)\mu + 6E(X^2)\mu^2 – 4\mu \mu^3 + \mu^4$$
$$\mu_4 = E(X^4) – 4E(X^3)E(X) + 6E(X^2)E(X)^2 – 3E(X)^4 \quad \text{or equivalently}$$
$$= \mu_4′ – 4\mu_3’\mu_1′ + 6\mu_2′(\mu_1′)^2 – 3(\mu_1′)^4$$
The second moment $\mu_2$ is called the variance of $X$ and it is denoted by $\sigma^2$, $\text{Var}(X)$ or simply $V(X)$. Also $+\sqrt{V(X)}$, positive square root of the variance, is called the standard deviation $(\sigma)$ of $X$ that is $\sigma = +\sqrt{V(X)}$. This measure is indicating the dispersion of the distribution of the random variable $X$.
Now we consider two more quantities which are defined as measure of skewness and measure of Kurtosis. Skewness is the degree of asymmetry of a distribution. Kurtosis is the degree of peakedness of a distributionmeasure of skewness is defined as $\gamma_{1} = (\mu_{3} ^ 2) / (\mu_{2} ^ 2)$ and a measure of Kurtosis is defined as $\beta_{2} = \mu_{4} / (\mu_{2} ^ 2)$.
$\gamma_{1}$ could be negative, zero, and a distribution. In that case the distribution is said to be skewed to the left, not skewed (symmetric) and skewed to the right, respectively. A distribution with a high peak $(\beta_{2} > 3)$ is called leptokurtic, a flat-topped curve $(\beta_{2} < 3)$ is called platykurtic and neither of these $(\beta_{2} = 3)$ is called the normal curve or mesokurtic.
Example 2
Find the mean and variance of following random variables.
$$ \text{(a)} \quad f_X(x) = \begin{cases} \dfrac{x+2}{14} & x = 0, 1, 2, 3 \\ 0 & \text{elsewhere} \end{cases} $$
$$ \text{(b)} \quad f_X(x) = \begin{cases} 3x^2 & 0 < x < 1 \\ 0 & \text{elsewhere} \end{cases} $$
(a) $A = \{0, 1, 2, 3\}$, hence $X$ is a discrete random variable. So we write its pmf as
$$ \begin{array}{c|cccc} X = x & 0 & 1 & 2 & 3 \\ \hline f_X(x) & \frac{1}{7} & \frac{3}{14} & \frac{2}{7} & \frac{5}{14} \end{array}$$
$$\mu_1′ = \mu = \sum x f_X(x) = 0 \cdot \frac{1}{7} + 1 \cdot \frac{3}{14} + 2 \cdot \frac{2}{7} + 3 \cdot \frac{5}{14} = \frac{15}{14} = \frac{13}{7}$$
$$ \mu_2′ = E(X^2) = \sum x^2 f_X(x) = 0 \cdot \frac{1}{7} + 1 \cdot \frac{3}{14} + 4 \cdot \frac{2}{7} + 9 \cdot \frac{5}{14} = \frac{15}{14} = \frac{32}{7}$$
$$ \therefore \quad V(x) = \mu_2′ – \mu^2 = \frac{32}{7} – \left(\frac{13}{7}\right)^2 = \frac{55}{49} $$
(b) Here $A = \{x/0 < x < 1\}$ hence $X$ is a continuous random variable.
$$ \mu_1′ = \int_{-\infty}^{\infty} xf_X(x) \, dx = \int_0^1 x \cdot 3x^2 \, dx = 3\left(\frac{x^4}{4}\right)_0^1 = \frac{3}{4} $$
$$ \mu_2′ = E(X^2) = \int_{-\infty}^{\infty} x^2 f_X(x) \, dx $$
$$ = \int_0^1 x^2 3x^2 \, dx = 3\left(\frac{x^5}{5}\right)_0^1 = \frac{3}{5} $$
$$ \therefore \quad V(X) = \mu_2′ – \mu^2 = \frac{3}{5} – \left(\frac{3}{4}\right)^2 = \frac{3}{80} $$
Example 3
Find the four moments for the following distributions.
$$ \text{(a)} \quad f_X(x) = \begin{cases} Kx & x = 1, 2, 3, 4 \\ 0 & \text{elsewhere} \end{cases} $$
$$ \text{(b)} \quad f_X(x) = \begin{cases} K & 1 < x < 4 \\ 0 & \text{elsewhere} \end{cases}$$
(a) In this case $A = \{1, 2, 3, 4\}$ and $X$ is a discrete RV.
Hence $\displaystyle\sum_{x} f_X(x) = 1 \implies K(1) + K(2) + K(3) + K(4) = 1$
$$\implies 10K = 1; \quad K = \frac{1}{10}$$
Hence pmf of $X$ is
$$ \begin{array}{c|cccc} X = x & 1 & 2 & 3 & 4 \\ \hline f_X(x) & \frac{1}{10} & \frac{2}{10} & \frac{3}{10} & \frac{4}{10} \end{array}$$
$$\mu = \mu_1′ = E(X) = \sum xf_X(x) = 1\left(\frac{1}{10}\right) + 2\left(\frac{2}{10}\right) + 3\left(\frac{3}{10}\right) + 4\left(\frac{4}{10}\right) = 3$$
$$\mu_2′ = E(X^2) = \sum x^2 f_X(x) = 1^2\left(\frac{1}{10}\right) + 2^2\left(\frac{2}{10}\right) + 3^2\left(\frac{3}{10}\right) + 4^2\left(\frac{4}{10}\right) = 10$$
$$\mu_3′ = E(X^3) = \sum x^3 f_X(x) = 1^3\left(\frac{1}{10}\right) + 2^3\left(\frac{2}{10}\right) + 3^3\left(\frac{3}{10}\right) + 4^3\left(\frac{4}{10}\right) = 35.4$$
$$\mu_4′ = E(X^4) = \sum x^4 f_X(x) = 1^4\left(\frac{1}{10}\right) + 2^4\left(\frac{2}{10}\right) + 3^4\left(\frac{3}{10}\right) + 4^4\left(\frac{4}{10}\right) = 130$$
$$\mu_1 = 0 \ (\text{Always})$$
$$\quad \mu_2 = \mu_2′ – \mu^2 = 10 – 3^2 = 1$$
$$\mu_3 = \mu_3′ – 3\mu_2’\mu + 2\mu^3 = 35.4 – 3(10)(3) + 2(3)^3 = -0.6$$
$$\mu_4 = \mu_4′ – 4\mu_3’X\mu + 6\mu_2’\mu^2 – 3\mu^4 = 130 – 4(35.4)(3) + 6(10)(3)^2 – 3(3)^4 = 2.2$$
(b) In this case $A = \{x/1 \leq x \leq 4\}$, $X$ is a continuous random variable
Hence
$$\int_{-\infty}^{\infty} f_X(x) \, dx = 1 \implies \int_1^4 K \, dx = 1 \implies K = \frac{1}{3}$$
$$ \therefore \quad f_X(x) = \begin{cases} \dfrac{1}{3} & 1 < x < 4 \\ 0 & \text{elsewhere} \end{cases}$$
$$\mu = \mu_1′ = E(X) = \int_{-\infty}^{\infty} xf_X(x) \, dx = \int_1^4 x\left(\frac{1}{3}\right) dx = \frac{1}{3}\left(\frac{x^2}{2}\right)_1^4 = \frac{5}{2}$$
$$\mu_2′ = E(X^2) = \int_{-\infty}^{\infty} x^2 f_X(x) \, d(x) = \int_1^4 x^2 \left(\frac{1}{3}\right) dx = 7$$
$$\mu_3′ = E(X^3) = \int_{-\infty}^{\infty} x^3 f_X(x) \, dx = \int_1^4 x^3 \left(\frac{1}{3}\right) dx = \frac{85}{4}$$
$$\mu_4′ = E(X^4) = \int_{-\infty}^{\infty} x^4 f_X(x) \, dx = \int_1^4 x^4 \left(\frac{1}{3}\right) dx = \frac{341}{5}$$
$$\mu_1 = 0 \ (\text{Always})$$
$$\mu_2 = \mu_2′ – \mu^2 = 7 – \left(\frac{5}{2}\right)^2 = \frac{3}{4}$$
$$\mu_3 = \mu_3′ – 3\mu_2’\mu + 2\mu^3 = \frac{85}{4} – 3(7)\left(\frac{5}{2}\right) + 2\left(\frac{5}{2}\right)^3 = 0$$
$$ \mu_4 = \mu_4′ – 4\mu_3’\mu + 6\mu_2’\mu^2 – 3\mu^4 $$
$$ = \frac{341}{5} – 4\left(\frac{85}{4}\right)\left(\frac{5}{2}\right) + 6(7)\left(\frac{5}{2}\right)^2 – 3\left(\frac{5}{2}\right)^4 = \frac{81}{80} $$