Introduction
In this note, we study how to find the probability distribution of functions of one or more random variables. That is, if $X$ is a random variable (discrete or continuous), we can define a new random variable $Y = g(X)$, which is a function of $X$. The main question is: if the probability density function (pdf) of $X$ is known, can we find the pdf of $Y$?
We may recall that if $X \sim N(\mu, \sigma^2)$, then the standardized variable
$$Z = \dfrac{X – \mu}{\sigma} \sim N(0,1),$$
which makes it easier to compute probabilities for the normal distribution. Similarly, while finding the expectation of a function of one or more random variables, we often use the pdfs of the variables and certain properties of expectations. However, for some transformations, this process can become difficult or impractical.
In such situations, we can use alternative approaches to find expectations or other properties of transformed variables. Several methods are available for solving this kind of problems.
In this note, we shall discuss three methods: the distribution function technique, the change-of-variable technique, and the moment-generating function technique. We will also produce two important distributions, which are useful in statistical inference, using these transformations.
DISTRIBUTION FUNCTION TECHNIQUE
If $X$ is a continuous random variable and has a distribution function $F_X(x)$, then by differentiating $F_X(x)$, we can obtain the pdf $f_X(x)$ , that is $f_X(x) = F_X'(x)$. This relationship between the distribution function and pdf of $X$, provides a straight forward method of obtaining the pdf of a function of the continuous random variable $X$.
Consider a 1-D continuous random variable $X$ with space $\mathscr{A}_X= \{x/f_X(x) > 0\}$ and $Y = g(X)$ is a function of the random variable $X$. Hence $Y$ itself is a random variable with its own space $\mathscr{B}_Y = \{y/\phi_Y(y) > 0\}$ where $\phi_Y(y)$ is the pdf of $Y$.
The Space $\mathscr{B}_Y $ can also be written as $\mathscr{B}_Y = \{y/y = g(x), x \in \mathscr{A}_X\}$ that is the points of $\mathscr{B}_Y $ are the images of the points of $\mathscr{A}_X$ under the transformation $Y = g(X)$. Now, let us apply the distribution function technique to obtain the pdf of $Y = g(X)$.
If $y \in B$ and $Y \le y$ which means $Y = g(X) \le y$ occurs only when the event $g(x) \le y$ for the points $x$ in some subset $A$ of $\mathscr{A}_X$. Hence if $G_Y(y)$ is the distribution function of $Y$ then
$$G_Y(y) = p[Y \le y] = p[g(X) \le y]$$
$$= p(A)$$
Now to obtain the pdf of $Y$, we have to differentiate $G_Y(y)$ with respect to $Y$, or
$$\phi_Y(y) = \dfrac{d}{dy} G_Y(y)$$
Let us summarize the above ideas as follows.
Identify the space $\mathscr{A}_X$ of $X$ and the transformation $Y = g(X)$.
Find the space $\mathscr{B}_Y$ of $Y$ by finding the images of the end points of $\mathscr{A}_X$ and write the appropriate relationship of the points of $\mathscr{B}_Y$.
Calculate the distributions function $F_X(x)$ of the random variable $X$.
Define $G_Y(y)$ as the distribution function as
$$G_Y(y) = p[Y \le y] = p[g(X) \le Y] = p[X \le g^{-1}(y)]$$
$$= F_X[g^{-1}(Y)]$$
Hence obtain the pdf of $Y$ as
$$\phi_Y(y) = \dfrac{d}{dy} G_Y(y).$$
Example 1
If the pdf of $X$ is
$$f_X(x) = \begin{cases} 6x(1-x) & 0 < x < 1 \\ 0 & \text{elsewhere} \end{cases}$$
find the pdf of $Y = X^2$.
Here $\mathscr{A}_X = \{x/0 < x < 1\}$ and $g(X) = X^2$. Let $\mathscr{B}_Y$ be the space of the random variable $Y$, so that its points are image of the points of $\mathscr{A}_X$. That is the image of ‘$0$’ in $\mathscr{A}_X$ is $g(0) = 0$ and the image of ‘$1$’ in $\mathscr{A}_X$ is $g(1) = 1$.
$$\therefore \quad \mathscr{B}_Y = \{y/0 < y < 1\}$$
If $F_X(x)$ is the distribution function then
$$F_X(x) = p(X \le x) = \int_0^x f_X(t)\, dt = \int_0^x 6t(1-t)\, dt$$
$$= 6\left(\dfrac{t^2}{2} – \dfrac{t^3}{3}\right)_0^x = 3x^2 – 2x^3.$$
Now if $G_Y(y)$ is the distribution function of $Y$ then $G_Y(y) = F_X(g^{-1}(y))$ where $g^{-1}(y) = +\sqrt{y}$ since $x > 0$ in $A$. Hence
$$G_Y(y) = 3(\sqrt{y})^2 – 2(\sqrt{y})^3$$
$$= 3y – 2y^{3/2} \quad \text{when } 0 < y < 1$$
that is
$$G_Y(y) = \begin{cases} 0 & y \le 0 \\ 3y – 2y^{3/2} & 0 < y < 1 \\ 1 & y \ge 1 \end{cases}$$
Differentiating $G_Y(y)$ with respect to $y$ we get the pdf of $Y$ as
$$\phi_Y(y) = \begin{cases} 3 – 3y^{1/2} & 0 < y < 1 \\ 0 & \text{elsewhere} \end{cases}$$
$$\phi_Y(y) = \begin{cases} 3(1-\sqrt{y}) & 0 < y < 1 \\ 0 & \text{elsewhere} \end{cases}$$
Note:
While finding the points of $\mathscr{B}_Y$ from the points of $\mathscr{A}_X$ and the from given transformation we depend on the end points of $\mathscr{A}_X$ and their corresponding images. But a careful analysis is required to from the relationship among the images of the end points of $\mathscr{A}_X$. That is the order in $\mathscr{A}_X$ need not be followed in the images of that end points.
Consider the following case, if $\mathscr{A}_X = \{x/0 < x < 1\}$ and $Y = g(X) = -2\log X$ then images are $g(0) = \infty$ (since $\log 0 = -\infty$) and $g(1) = 0$. Hence $\mathscr{B}_Y = \{y/0 < y < \infty\}$. Observe that the images of the end points of $\mathscr{A}_X$ namely $\infty$ and $0$ are reversed in order while we write $\mathscr{B}_Y$.
This appropriate ordering is necessary while we write the limits for points of $\mathscr{B}_Y$ as the image set of $\mathscr{A}_X$ under $Y = g(X)$. This point has to be applied carefully throughout this section as well as in our next section.
Example 2
If $X \sim \text{Uniform}(-1,1)$ find the pdf of $Y = |X|$.
$X \sim \text{Uniform}(-1,1)$ hence its pdf is
$$f_X(x) = \begin{cases} \dfrac{1}{2} & -1 < x < 1 \\ 0 & \text{elsewhere} \end{cases} \quad \text{and} \quad \mathscr{A}_X = \{x/-1 < x < 1\}$$
Given $Y = |X|$. So that $\mathscr{B}_Y = \{y/0 < y < 1\}$, (since $y = |X| \ge 0$ always the image of the end points of $\mathscr{A}_X$ decides only the upper limits of $\mathscr{B}_Y$).
Now
$$F_X(x) = \int_{-1}^{x} f_X(t)\, dt = \int_{-1}^{x} \dfrac{1}{2}\, dt$$
$$= \dfrac{1}{2}(x+1)$$
$$\therefore \quad G_Y(y) = p(Y \le y) = p[|X| \le y]$$
$$= p[-y \le X \le y]$$
$$= F[y] – F[-y] \quad \text{(Using a property of the distribution function)}$$
$$= \dfrac{1}{2}(y+1) – \dfrac{1}{2}(-y+1) = y.$$
Hence
$$G_Y(y) = \begin{cases} 0 & y \le 0 \\ y & 0 < y < 1 \\ 1 & y > 1 \end{cases}$$
Therefore the pdf $\phi_Y(y)$ of $Y$ is
$$\phi_Y(y) = \begin{cases} 1 & 0 < y < 1 \\ 0 & \text{elsewhere} \end{cases}$$
That is $Y \sim \text{Uniform}(0,1)$.
CHANGE OF VARIABLE TECHNIQUE
Let $X$ be a one dimensional capital discrete random variable with pmf $f_X(x)$ and space $\mathscr{A}_X$ that is $\mathscr{A}_X = \{x/f_X(x) > 0\}$. Let $y = g(X)$ be a one-to-one transformation that maps $\mathscr{A}_X$ onto $\mathscr{B}_Y$ where $\mathscr{B}_Y$ obtained by transforming each point in $\mathscr{A}_Y$ in accordance with $Y = g(X)$.
The one-to-one transformation is such that to each point in $\mathscr{A}_X$ there corresponds one, and only one, point in $\mathscr{B}_Y$; and conversely, to each point in $\mathscr{B}_Y$ there corresponds one, and only one, point in $\mathscr{A}_X$. If we solve $y = u(x)$ for $x$ in terms of $y$, say $x = w(y)$ then for each $y$ belongs to $\mathscr{B}_Y$, we have $x = W(y)$ belongs to $\mathscr{A}_X$.
Now considering $Y = g(x)$ as a random variable, the events $y = y$ and $x = W(y)$ are equivalent. Hence if we define $\phi_Y(y)$ as the pdf of the random variable $y$ then
$$\phi_Y(y) = \begin{cases} f_Y(w(y)) & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
The following example illustrates the idea so far we have discussed.
Example 3
Let $X$ have a pmf
$$f_X(x) = \begin{cases} \dfrac{1}{3} & x = 1,2,3 \\ 0 & \text{elsewhere} \end{cases}$$
Find the pdf of $Y = X^3$.
$\mathscr{A}_X = {1,2,3}$, $Y = X^3$ and hence $\mathscr{B}_Y = {1,8,27}$. $X = Y^{1/3}$. If $\phi_Y(y)$ is the pmf of $Y$ then,
$$\phi_Y(y) = \begin{cases} f_Y(w(y)) & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} \dfrac{1}{3} & y = 1,8,27 \\ 0 & \text{elsewhere} \end{cases}$$
Example 4
A random variable $X$ assumes 1, 2 and 3 with probabilities $\dfrac{1}{3}$, $\dfrac{1}{2}$ and $\dfrac{1}{6}$ respectively. Find the pmf of $y = 3X – 1$.
pmf of $X$ can be written as,
$$ \begin{array}{|c|c|c|c|} \hline X = x & 1 & 2 & 3 \\ \hline f_X(x) & \dfrac{1}{3} & \dfrac{1}{2} & \dfrac{1}{6} \\ \hline \end{array}$$
$\mathscr{A}_X = {1,2,3}$ and $g(X) = Y = 3X – 1$. Hence $\mathscr{B}_Y = {2,5,8}$ and $p(Y=2) = p(X=1) = \dfrac{1}{3}$ and so on. The pmf of $Y$ is
$$ \begin{array}{|c|c|c|c|} \hline Y = y & 2 & 5 & 8 \\ \hline \phi_Y(y) & \dfrac{1}{3} & \dfrac{1}{2} & \dfrac{1}{6} \\ \hline \end{array} $$
Transformations of variables of the continuous type:
Let $X$ be a random variable of the continuous type having pdf $f_X(x)$. Let $\mathscr{A}_X$ be the space of $X$ such that $\mathscr{A}_X = \{x/f_X(x) > 0\}$. Consider the random variable $Y = g(X)$ which defines a one-to-one transformation that maps the set $\mathscr{A}_X$ onto the set $\mathscr{B}_Y$ where $\mathscr{B}_Y$ is the space of $Y$ so that $\mathscr{B}_Y = \{y/\phi_Y(y) > 0\}$ and $\phi_Y(y)$ is the pdf of $Y$. Let the inverse of $Y = g(x)$ be $x = w(y)$. Then the pdf of $Y$ is given by
$$\phi_Y(y) = \begin{cases} f_Y[w(y)]\,|J| & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
where $J = \dfrac{dx}{dy} = w'(y)$ is continuous and non zero for all points $y$ in $\mathscr{B}_Y$. It will be called the Jacobian of the transformation.
Example 5
Let $X$ have the pdf
$$f_X(x) = \begin{cases} 3x^2 & 0 < x < 1 \\ 0 & \text{elsewhere} \end{cases}$$
Find the pdf of $Y = X^3$.
Here $\mathscr{A}_X = \{x/0 < x < 1\}$ and the pdf
$$f_X(x) = \begin{cases} 3x^2 & 0 < x < 1 \\ 0 & \text{elsewhere} \end{cases}$$
$Y = g(x) = X^3$
$$\therefore \quad X = Y^{1/3} = w(y) \quad \text{and} \quad \mathscr{B}_Y = \{y/0 < y < 1\}$$
$$J = \dfrac{dx}{dy} = \dfrac{1}{3}y^{-2/3} = \dfrac{1}{3y^{2/3}}$$
$$|J| = \dfrac{1}{3y^{2/3}}.$$
If $\phi_Y(y)$ is the pdf of $Y$, then
$$\phi_Y(y) = \begin{cases} f_Y(w(y))\,|J| & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} 3(y^{1/3})^2 \cdot \dfrac{1}{3y^{2/3}} & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} 1 & 0 < y < 1 \\ 0 & \text{elsewhere} \end{cases}$$
$$\therefore \quad Y \sim \text{Uniform}(0,1)$$
Example 6
Let $X \sim \text{Uniform}(0,1)$. Find the pdf of $Y = -2\log X$.
Given $X \sim \text{Uniform}(0,1)$
Therefore its pdf is
$$f_X(x) = \begin{cases} 1 & 0 < x < 1 \\ 0 & \text{elsewhere} \end{cases}$$
and $\mathscr{A}_X = \{x/0 < x < 1\}$ and $Y = g(x) = -2\log x$ so that $\mathscr{B}_Y = \{Y/0 < Y < \infty\}$ [observe that $g(0) = \infty$ and $g(1) = 0$] and $x = w(y) = e^{-y/2}$
$$\therefore \quad J = \dfrac{dx}{dy} = -\dfrac{1}{2}e^{-y/2}$$
$$|J| = \dfrac{1}{2}e^{-y/2}$$
If $\phi_Y(y)$ is the pdf of $Y$ then
$$\phi_Y(Y) = \begin{cases} f_Y(w(y))\,|J| & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} 1 \cdot \dfrac{1}{2}e^{-y/2} & y > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} \dfrac{1}{2}e^{-y/2} & y > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$$\therefore \quad y \sim \text{Exponential}\left(\dfrac{1}{2}\right).$$
So far we have considered our examples carefully so that our transformations $y$ are single-valued functions of $x$ and indeed the corresponding inverse transformations ‘$x$’ are also single-valued functions of $Y$ that is $Y = g(X)$ is such that to each point in $\mathscr{A}_X$ there correspondence one and only one point in $\mathscr{B}_Y$ and conversely $X = w(y)$ is such that to each point in $\mathscr{B}_Y$ there corresponds one and only one point in $\mathscr{A}_X$. But such cases need not exist always in our transformations.
For example if $X \sim \text{Uniform}(-1,1)$ and if we wish to find the pdf of $Y = X^2$ then $\mathscr{A}_X = \{x/-1 < x < 1\}$ and $\mathscr{B}_Y = \{y/0 \le y < 1\}$ so that to each point of $\mathscr{A}_X$ there will correspond one point in $\mathscr{B}_Y$ but to a (indeed 2 points) point in $\mathscr{B}_Y$ (of course, other than zero.) there corresponds more than one point in $\mathscr{A}_X$. For example $\dfrac{1}{4}$ in $\mathscr{B}_Y$ may correspond to $\dfrac{1}{2}$ and $\dfrac{-1}{2}$ in $\mathscr{A}_X$. That is, the transformation may not be one-to-one. Our next section deals this idea.
Hence if the given transformation is not one-to-one then the space $\mathscr{A}_X$ of $X$ is partitioned as $\mathscr{A}_1, \mathscr{A}_2 \cdots \mathscr{A}_k$ such that to each point in $\mathscr{B}_Y$ there will correspond exactly one point in each of $\mathscr{A}_1, \mathscr{A}_2 \cdots \mathscr{A}_k$, that is $\mathscr{A}_X = \mathscr{A}_1 \cup \mathscr{A}_2 \cup \cdots \cup \mathscr{A}_k$ and $\mathscr{A}_1 \cap \mathscr{A}_2 \cap \cdots \cap \mathscr{A}_k = \phi$ so that the desired one-to-one transformation is obtained. Then if $\phi_Y(y)$ is that pdf of $Y$ then it is given as
$$\phi_Y(y) = \sum_{i=1}^{k} |J_i|\, f_Y[w_i(y)]$$
where $f_X(x)$ is the pdf of the given random variable $X$ and $w_i(y)$ is the inverse transformation corresponding to the partition $\mathscr{A}_i$ and $J_i$ is the corresponding Jacobian. More precisely, to each point $y$ in $\mathscr{B}_Y$ there corresponds exactly one point in each of $\mathscr{A}_1, \mathscr{A}_2, \cdots \mathscr{A}_k$ as
$$x_1 = w_1(y),\ x_2 = w_2(y) \cdots x_k = w_k(y)$$
then
$$J_1 = \dfrac{dw_1}{dy},\ J_2 = \dfrac{dw_2}{dy}, \cdots, J_k = \dfrac{dw_k}{dy}$$
This discussion is illustrated with the help of the following example.
Example 7
If
$$f_X(x) = \begin{cases} \dfrac{1}{2} & -1 < x < 1 \\ 0 & \text{elsewhere} \end{cases}$$
is the pdf of the random variable $X$, find the pdf of $Y = X^2$.
Here $\mathscr{A}_X = \{x/-1 < x < 1\}$ and $Y = g(X) = X^2$ which is not one-to-one from $\mathscr{A}_X$ onto $\mathscr{B}_Y$ where $\mathscr{B}_Y = \{y/0 \le y < 1\}$. Now partition the set $\mathscr{A}_X$ as $\mathscr{A}_1 = \{x/-1 < x < 0\}$ and $\mathscr{A}_2 = \{x/0 \le x < 1\}$. Hence $Y = X^2$ with the inverse $x = -\sqrt{y}$ maps $\mathscr{A}_1$ onto $\mathscr{B}_Y$ and $x = +\sqrt{y}$ maps $\mathscr{A}_2$ onto $\mathscr{B}_Y$ and hence the transformation is one-to-one.
$$\therefore \quad x_1 = w_1(y) = -\sqrt{y} \quad \text{and} \quad x_2 = w_2(y) = \sqrt{y}$$
$$\therefore \quad J_1 = \dfrac{dw_1}{dy} = -\dfrac{1}{2\sqrt{y}};\quad J_2 = \dfrac{dw_2}{dy} = \dfrac{1}{2\sqrt{y}}$$
Hence if $\phi_Y(y)$ is the pdf of $Y$ then
$$\phi_Y(y) = \begin{cases} \left|\dfrac{-1}{2\sqrt{y}}\right| \cdot \dfrac{1}{2} + \left|\dfrac{1}{2\sqrt{y}}\right| \cdot \dfrac{1}{2} & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} \dfrac{1}{2\sqrt{y}} & 0 < y < 1 \\ 0 & \text{elsewhere} \end{cases}$$
Example 8
Let $X \sim \text{Normal}(0,1)$ find the pdf of $Y = X^{(2)}$.
Here $f_X(x)$, the pdf of $X$ is
$$f_X(x) = \dfrac{1}{\sqrt{2\pi}}e^{-x^2/2} \qquad -\infty < x < \infty$$
so that $\mathscr{A}_X = \{x/-\infty < x < \infty\}$. Given that $Y = X^2$ hence $\mathscr{B}_Y = \{y/y \ge 0\}$. Let us partition $\mathscr{A}_X$ as $\mathscr{A}_X = \mathscr{A}_1 \cup \mathscr{A}_2$ where $\mathscr{A}_1 = \{x/-\infty < x < 0\}$ and $\mathscr{A}_2=\{x/0 \le x < \infty\}$. Now $x_1 = -\sqrt{y}$ and $x_2 = \sqrt{y}$ are the inverse transformation so that
$$J_1 = -\dfrac{1}{2\sqrt{y}} \quad \text{and} \quad J_2 = \dfrac{1}{2\sqrt{y}}$$
Hence if $\phi_Y(y)$ is the pdf of $Y$ then,
$$\phi_Y(y) = \begin{cases} \left|-\dfrac{1}{2\sqrt{y}}\right|\dfrac{1}{\sqrt{2\pi}}e^{-y/2} + \left|\dfrac{1}{2\sqrt{y}}\right|\dfrac{1}{\sqrt{2\pi}}e^{-y/2} & y \in \mathscr{B}_Y \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} \dfrac{1}{\sqrt{2\pi}}\cdot\dfrac{1}{2\sqrt{y}}\cdot 2e^{-y/2} & y > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$$\phi_Y(y) = \begin{cases} \dfrac{1}{\sqrt{2\pi y}}e^{-y/2} & y > 0 \\ 0 & \text{elsewhere} \end{cases}$$
Extension of the Change-of-Variable Techniques:
In our previous section, it was seen that the determination of the pdf of a function of a random variable of the discrete or continuous type in one dimensional case. Now, let us extend this idea to the two dimensional cases (both discrete and continuous) as a natural generalization of our previous section.
2-D Discrete cases:
Let $f_{XY}(x,y)$ be the joint p.m.f of two discrete random variable $X$ and $Y$ with the space $\mathscr{A}_{XY} = \{(x,y)/f_{XY}(x,y) > 0\}$. Let $u = g_1(x,y)$ and $v = g_2(x,y)$ define a one-to-one transformations that map $\mathscr{A}_{XY}$ onto $\mathscr{B}_{UV}$ where $u$ and $v$ are auxiliary random variables. The single valued inverse of $u = g_1(x,y)$ and $v = g_2(x,y)$ are calculated as $x = w_1(u,v)$ and $y = w_2(u,v)$. Then if $\phi_{UV}(u,v)$ is the joint pmf of $u$ and $v$ then
$$\phi_{UV}(u,v) = \begin{cases} f_{UV}[w_1(u,v),\ w_2(u,v)] & (u,v) \in \mathscr{B}_{UV} \\ 0 & \text{elsewhere} \end{cases}$$
where $\mathscr{B}_{UV}$ is the space of the two auxiliary random variables $u$ and $v$ such that $\mathscr{B}_{UV} = \{(u,v)/\phi_{UV}(u,v) > 0\}$.
2-D Continuous case:
The method of finding the pdf of a function of one random variable of the continuous type will now be extended to functions of two random variables of the continuous type. Again, only one-to-one transformation will be considered. Otherwise the idea of partitioning the space $\mathscr{A}_{XY}$ can be extended in an appropriate sense.
Anyway, at this time, we assume $u = g_1(x,y)$ and $v = g_2(x,y)$ be a one-to-one transformation that maps $\mathscr{A}_{XY}$ (Space of $(X,Y)$) onto $\mathscr{B}_{UV}$ (Space of $(u,v)$), that is a set in the $XY$ plane onto a set in the $UV$-plane. With the usual notations for inverse transformations we have the joint pdf of $u$ and $v$ as
$$\phi_{UV}(u,v) = \begin{cases} f_{UV}[w_1(u,v),\ w_2(u,v)]\cdot|J| & (u,v) \in \mathscr{B}_{UV} \\ 0 & \text{elsewhere} \end{cases}$$
where
$$J = \dfrac{\partial(x,y)}{\partial(u,v)} = \begin{vmatrix} \dfrac{\partial x}{\partial u} & \dfrac{\partial x}{\partial v} \\ \dfrac{\partial y}{\partial u} & \dfrac{\partial y}{\partial v} \end{vmatrix}$$
is called the Jacobian of the transformation and it will be assumed that $J \ne 0$ in $\mathscr{B}_{UV}$.
Note:
It should be emphasized that the technique of change of variables (in 2 variables cases) involves two functions $U$ and $V$ of $x, y$, where $U$ and $V$ are auxiliary random variables. However, in some cases this technique involves the introduction of a new variable ($V$) if we are interested to find the pdf of only one variable say $U$ as a function $X$ and $Y$. In such cases, we can find the joint pdf of $U$ and $V$ and from that we can find the marginal pdf of $U$. (or $v$, of course) by appropriate integration or summation.
Example 9
If the joint pdf of $X$ and $Y$ is given by
$$f_{XY}(x,y) = \begin{cases} \dfrac{xy}{36} & x = 1,2,3;\ y = 1,2,3 \\ 0 & \text{elsewhere} \end{cases}$$
Find the joint pdf of $X+Y$ and $X-Y$.
Given that $\mathscr{A}_{XY} = \{(x,y)/x=1,2,3\ \quad y = 1,2,3\}$. Therefore $X$ and $Y$ are discrete random variables.
$$\begin{array}{|c|c|c|c|} \hline X\backslash Y & 1 & 2 & 3 \\ \hline 1 & \dfrac{1}{36} & \dfrac{2}{36} & \dfrac{3}{36} \\ \hline 2 & \dfrac{2}{36} & \dfrac{4}{36} & \dfrac{6}{36} \\ \hline 3 & \dfrac{3}{36} & \dfrac{6}{36} & \dfrac{9}{36} \\ \hline \end{array} $$
If $\mathscr{B}_{UV}$ is the space of $u = X+Y$ and $v = X-Y$ then
$$\mathscr{B}_{UV} = {(u,v)/(4,-2);\ (5,-1);\ (3,-1);\ (2,0);\ (4,0);\ (6,0);\ (3,1);\ (5,1);\ (4,2)}$$
$$\begin{array}{|c|c|c|c|c|c|} \hline u \backslash v & -2 & -1 & 0 & 1 & 2 \\ \hline 2 & 0 & 0 & \dfrac{1}{36} & 0 & 0 \\ \hline 3 & 0 & \dfrac{2}{36} & 0 & \dfrac{2}{36} & 0 \\ \hline 4 & \dfrac{3}{36} & 0 & \dfrac{4}{36} & 0 & \dfrac{3}{36} \\ \hline 5 & 0 & \dfrac{6}{36} & 0 & \dfrac{6}{36} & 0 \\ \hline 6 & 0 & 0 & \dfrac{9}{36} & 0 & 0 \\ \hline \end{array}$$
$$p[(u,v) = (4,-2)] = p[X+Y=4 \text{ and } X-Y = -2]$$
$$= p[X=1 \text{ and } Y=3] = \dfrac{3}{36}$$
$$p[(u,v) = (5,-1)] = p[X+Y=5,\ X-Y=-1]$$
$$= p[X=2\ Y=3]$$
In a similar way other probabilities can be computed (Refer jpmf table of $(u,v)$)
Hence the pmf of $u$ and $v$ can be computed from the jpmf of $(u,v)$ as follows:
$$\begin{array}{|c|c|c|c|c|c|} \hline U=u & 2 & 3 & 4 & 5 & 6 \\ \hline p[U=u] & \dfrac{1}{36} & \dfrac{4}{36} & \dfrac{10}{36} & \dfrac{12}{36} & \dfrac{9}{36} \\ \hline \end{array} \quad \begin{array}{|c|c|c|c|c|c|} \hline V=v & -2 & -1 & 0 & 1 & 2 \\ \hline p[V=v] & \dfrac{3}{36} & \dfrac{8}{36} & \dfrac{14}{36} & \dfrac{8}{36} & \dfrac{3}{36} \\ \hline \end{array} $$
Example 10
If the joint density of $X$ and $Y$ is
$$f_{XY}(x,y) = \begin{cases} e^{-(x+y)} & x>0;\ y>0 \\ 0 & \text{elsewhere} \end{cases}$$
find the joint density of $u = X+Y$ and $v = \dfrac{X}{X+Y}$.
Here $\mathscr{A}_{XY} = \{(x,y)/x>0;\ y>0\}$.
$$u = X+Y \qquad v = \dfrac{X}{X+Y}$$
Hence
$$v = X/u $$
$$\Rightarrow X = uv \quad \text{and} \quad Y = u-X$$
$$= u-uv$$
$$Y = u(1-v)$$
$$\therefore \quad J = \dfrac{\partial[x, y]}{\partial(u, v)} = \begin{vmatrix} \dfrac{\partial x}{\partial u} & \dfrac{\partial x}{\partial v} \\ \dfrac{\partial y}{\partial u} & \dfrac{\partial y}{\partial v} \end{vmatrix} = \begin{vmatrix} v & u \\ 1-v & -u \end{vmatrix}$$
$$= -vu – u(1-v) = -u.$$
$$\therefore \quad |J| = u$$
To find $\mathscr{B}_{UV}$, the space of the 2-D random variable $(u, v)$, since $x > 0$, $y > 0$ we have, $u > 0$ and $x + y > x$ so that $0 < v < 1$.
Therefore the space of $(u, v)$ is
$$\mathscr{B}_{UV} = \{(u, v)/u > 0 \text{ and } 0 < v < 1\}$$
If $\phi_{UV}(u, v)$ is the joint pdf of $(u, v)$ then
$$\phi_{UV}(u, v) = \begin{cases} e^{-u}\cdot u & (u, v) \in \mathscr{B}_{UV} \\ 0 & \text{elsewhere} \end{cases}$$
$$= \begin{cases} u e^{-u} & u > 0 \text{ and } 0 < v < 1 \\ 0 & \text{elsewhere} \end{cases}$$
Note:
In the previous example if we wish to find the pdf of $u = x + y$ or the pdf of $v = \dfrac{x}{x+y}$ then we can integrate $\phi_{UV}(u, v)$ w.r. to $v$ and $u$ respectively that is $g(u)$ is the pdf of $u$ then
$$g_U(u) = \int_{-\infty}^{\infty} \phi_{UV}(u, v)\,dv = \int_0^1 u e^{-u}\,dv$$
$$g_U(u) = \begin{cases} u e^{-u} & u > 0 \\ 0 & \text{elsewhere} \end{cases}$$
Similarly if $h_V(v)$ is the pdf of $v$ then
$$h_V(v) = \int_{-\infty}^{\infty} \phi_{UV}(u, v)\,du = \int_0^{\infty} u e^{-u}\,du = \Gamma2 = 1.$$
$$\therefore \quad h_V(v) = \begin{cases} 1 & 0 < v < 1 \\ 0 & \text{elsewhere} \end{cases}$$
MOMENT-GENERATING FUNCTION TECHNIQUE
We should recall that a moment-generating function, if it exists, is unique and it uniquely determines the distribution of a probability. This property of mgf of a random variable is useful in building an alternative procedure of finding the distribution of a function of a random variable or a function of several random variables.
Recall that the reproductive (additive) property of certain special distributions, such as the binomial and Poisson distributions, has been established in earlier notes (see Theoretical Distributions). The method adopted therein is essentially similar to the one we shall discuss now. In that context, we considered only one form of a function of several random variables, namely the sum of independent random variables. However, this approach can be extended to other functions as well. We begin by introducing this idea for a function of a single random variable.
Let $f_X(x)$ be the pdf of a random variable $X$. Let $y = g(x)$ be a function of $X$. Consider the moment-generating function of $Y$. If it exists, it is given by
$$M_Y(t) = E[e^{tY}] = E[e^{tg(x)}]$$
Also if the moment-generating function of $Y$ is seen to be that of a certain kind of distribution, the uniqueness property makes it certain that $Y$ has that kind of distribution. We shall now give some examples where we use the moment-generating function technique.
Example 11
Let $X \sim \text{Normal}(0,1)$ show that $Y = X^2$ is a chi-square distribution with $r=1$.
Given $X \sim \text{Normal}(0,1)$ hence its pdf is
$$f_X(x) = \dfrac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}$$
and its space is $\mathscr{A}_X = (-\infty,\infty)$. Hence the space of $Y = X^2$ is $\mathscr{B}_Y = \{y \in \mathbb{R}/y > 0\} = (0,\infty)$
Now
$$M_Y(t) = E[e^{tY}] = E[e^{tX^2}] = \int_{-\infty}^{\infty} e^{tx^2}f_X(x)\,dx$$
$$= \int_{-\infty}^{\infty} e^{tx^2}\dfrac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}\,dx$$
$$= \dfrac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-x^2\left(\frac{1-2t}{2}\right)}\,dx$$
Substituting $w = x\sqrt{\dfrac{1-2t}{2}}$ we have,
$$dx = \dfrac{\sqrt{2}}{\sqrt{1-2t}}\,dw$$
and
$$M_Y(t) = \dfrac{1}{\sqrt{2\pi}}\cdot\dfrac{\sqrt{2}}{\sqrt{1-2t}}\int_{-\infty}^{\infty} e^{-w^2}\,dw$$
Again we write $t = w^2$ so that $dt = 2w\,dw$ and use the fact that the integrand is an even function of $w$, we have
$$M_Y(t) = \dfrac{1}{\sqrt{\pi}}\cdot\dfrac{1}{\sqrt{1-2t}}\int_0^{\infty} t^{-\frac{1}{2}}e^{-t}\,dt$$
$$= \dfrac{1}{\sqrt{\pi}}\cdot\dfrac{1}{\sqrt{1-2t}}\Gamma\frac{1}{2} = \dfrac{1}{(1-2t)^{1/2}}$$
which is the mgf of a chi-square distribution with parameter $r=1$ (Refer Theoretical Distribution notes).
Hence $Y \sim \pi^2(1)$.
$\textbf{Note:}$ Compare this method with Example 8
THE $t$ AND $F$ DISTRIBUTIONS
lets see two additional distributions called ‘$t$’ and ‘$F$’ distributions which are quite useful in statistics. We shall define them and their pdfs by the techniques of transformation of variables. However in our further discussion we are not much concerned with the applications of $t$ and $F$ distributions.
Result
1. Let $W \sim \text{Normal}(0,1)$ and $V \sim \text{Chi-square}(r)$ and let $W$ and $V$ be independent random variables. Then $T = \dfrac{W}{\sqrt{V/r}}$ is said to have (student’s) $t$ distribution with ‘$r$’ degrees of freedom.
Using the pdfs of $\text{Normal}(0,1)$ and $\text{Chi-square}(r)$ the jpdf of $W$ and $V$ is
$$f_{WV}(w,v) = \begin{cases} \dfrac{1}{\sqrt{2\pi}}e^{-\frac{w^2}{2}}\ \cdot\ \dfrac{1}{2^{\frac{r}{2}}\Gamma\frac{r}{2}}e^{-\frac{v}{2}}v^{\frac{r}{2}-1} & -\infty < w < \infty,\ v>0 \\ 0 & \text{elsewhere} \end{cases}$$
2.Let $X$ and $Y$ be two independent random variables such that
$$X \sim \text{Chi-square}(r_1)$$
and
$$Y \sim \text{Chi-square}(r_2)$$
then
$$F = \dfrac{X/r_1}{Y/r_2}$$
follows a distribution which is called F-distribution with two parameters $r_1$ and $r_2$.
Since $X \sim \text{Chi-square}(r_1)$ and $Y \sim \text{Chi-square}(r_2)$ and they are independent the jpdf of $X$ and $Y$ is
$$f_{XY}(x,y) = \begin{cases} \dfrac{1}{\Gamma\dfrac{r_1}{2} \Gamma\dfrac{r_2}{2}2^{\frac{r_1+r_2}{2}}}x^{\frac{r_1}{2}-1}y^{\frac{r_2}{2}-1}e^{-\left(\frac{x+y}{2}\right)} & x,y \ge 0 \\ 0 & \text{elsewhere} \end{cases}$$