Introduction
We generalise the concept of probability distribution of a random variable to the joint distribution of two random variables. The need of such generalization is often experiments are conducted where two random variables are observed simultaneously in order to determine not only their individual behaviour but also the degree of relationship between them.
keywords:
Probability function
let X and Y be a two random variables Whose space be $\mathscr{A}_X$ and $\mathscr{A}_Y$ respectively. let $\mathscr{A}_{XY} =\{(x,y)/ x \in \mathscr{A}_X ~\text{and} ~y \in \mathscr{A}_Y\}$. then $\mathscr{A}_{XY}$ is the space of the random variable $(X,Y)$ in $\mathbb{R}^2$ $\mathscr{A}_{XY}$ is finite or infinite accordingly as $\mathscr{A}_X$ and $\mathscr{A}_Y$ are finite or infinite.
Joint Probability Density Function (JPDF):
If $f_{XY}(x,y)$ is a function defined on $\mathscr{A}_{XY}$ is defined as the pdf of the random variable $(X,Y)$ if it satisfies the two conditions
- non-negative
- Total probability is one
we define it more formally as follows:
- $f_{XY}(x, y) \ge 0$
- $\sum_{\mathscr{A}_X}\sum_{\mathscr{A}_Y} f_{XY}(x,y)=1$ or $\int_{\mathscr{A}_X}\int_{\mathscr{A}_Y}f_{XY}(x,y)~~dx~dy=1$
Note that summation or integration over $\mathscr{A}_x$ & $\mathscr{A}_y$ depends on the nature of the variable $X$ & $Y$ as Discrete or Continuous.
At this point we can compare the definition of pdf of a 1-dimensional or 2-dimentional random variable (discrete or continuous), essentially it satisfies the conditions of being a pdf (a) $f$ is defined and is not negative for all real values of its arguments, (b) its integral or sum (according to continuous or discrete) over all real values of arguments is 1. This idea will help us to generalise or extend the notion of pdf of a random variable to even more than 2 variables.
Example 1:
Three coins are tossed. Let $X$ be the numberof heads on the first two coins. $Y$ be the number of tails on the last two. Let us form the joint pdf of $X$ and $Y$.
Spaces of $X$ and $Y$ are $\mathscr{A}_X$= $\mathscr{A}_Y$ = $\{0,1,2\}$ (Remember the sample space if the experiment continuous 8 points).
$\therefore$ the space of $\mathscr{A}_{XY} = \mathscr{A}_{X} \times \mathscr{A}_{Y} = \{(x,y)/ x = 0, 1, 2; ~\text{and} ~y= 0, 1, 2\}$
For instance, $$f_{XY}(1,2) = p(x=1,y=2)$$
$$= p (\text{Number of heads on the first two coins =1 and number of tails in the last two coins=2})$$
$$=\frac{1}{8} (\text{HTT})$$
Similarly, $$f_{XY}(1,1) = \frac{2}{8} (\text{HTH, THT})$$
Note that the choice of the value of $X$ and $Y$ should be made correctly to understand the joint probabilities of $X$ and $Y$.
Example 2:
In this example let us consider a bivariate pdf in continuous case. If we wish to select a point randomly from inside a circle $x^2 + y^2 = 25$ ,let us determine the joint pdf of $X$ and $Y$.
Here $\mathscr{A}_X=[-5,5]$ and $\mathscr{A}_Y=[-5,5]$
$$\therefore \mathscr{A}_{XY} = \mathscr{A}_{X} \times \mathscr{A}_{Y} \text{ and interior of the circle } x^2 + y^2 = 25$$
$$=\{(x,y)/ -5 \le x \le 5, -5 \le y \le 5\text{ and }x^2 + y^2 = 25\}$$
is the space of $(X,Y)$ which represents a random point inside the circle $x^2 + y^2 = 25$.
i.e.. the joint pdf of $X$ and $Y$ is constant over $\mathscr{A}_{XY}$ and $0$ outside.
i.e.. $$f_{XY}(x,y)=\begin{cases} C & (x,y)\in\mathscr{A}_{XY} \\ 0 & elsewhere \end {cases}$$
but we must have
$$\iint\limits_{\mathscr{A}_{XY}} f_{XY}(x,y)dx dy=1$$
$$\Rightarrow C \iint\limits_{\mathscr{A}_{XY}} f_{XY}(x,y)dx dy=1$$
$$\Rightarrow C (\text{Area of the circle)}=1$$
$$\Rightarrow C. 25\pi = 1$$
$$\Rightarrow C = \frac{1}{25\pi}$$
$$\therefore f_{XY}(x,y) = \begin{cases} \frac{1}{25\pi} & |x|\le5,|y|\le25 \text{ and } x^2 + y^2 = 25 \\ 0 & \text{elsewhere} \end{cases}$$
Marginal Probability Functions:
Let $f_{XY}(x,y)$ be the pdf of two random variables $X$ and $Y$. By assuming $X$ and $Y$ as random variables in one variables they have pdfs on their own satisfying the two conditions. This pdfs can be found from the joint pdf of $(X,Y)$. Such pdfs are said to be marginal pdf of $X$ and that of $Y$. we define them as follows.
MPDF of $X$:
$$ f_X(x)=\begin{cases} \int_{-\infty}^{\infty} f_{XY}(x,y) dy & \text{for the continuous case} \\ \sum_y f_{XY}(x,y) & \text{for the discrete case} \end{cases}$$
similarly MPDF of $Y$ is:
$$ f_Y(y)= \begin{cases} \int_{-\infty}^{\infty} f_{XY}(x,y) dx & \text{for the continuous case} \\ \sum_x f_{XY}(x,y) & \text{for the discrete case} \end{cases}$$
Conditional Distributions:
we shall now define the notion of a conditional pdf. Let $X$ and $Y$ denote random variables which have the joint pdf $f_{XY}(x,y)$ with space $\mathscr{A}_{XY}$. Let $f_X(x), f_Y(y)$ be their marginal probability density function respectively. Then the conditional porbability that $Y=y$ given that $X=x$ is defined as
$$f_{X|Y}(x|y)=\frac{f_{XY}(x,y)}{f_Y(y)}$$
where $f_X(x)>0$.
In a similar way of conditional probability we can define the conditional pdf of $X=x$ given that $Y=y$ as
$$f_{Y|X}(y|x)=\frac{f_{XY}(x,y)}{f_X(x)}$$
provided $f_Y(y) > 0$.
Depending upon the narure of the random variable $(X, Y)$ as discrete or continuous random variable, the computation of $f_X(x)$ and $f_Y(y)$ will be summation or integration.
Independent random variables:
Two random variables $X$ and $Y$, defined on the same probability space, are said to be independent if the realization of $X$ has no bearing on the realization of $Y$, and vice-versa.
Formally, this means that any information regarding the value assumed by $X$ does not affect the information regarding the value assumed by $Y$, and, symmetrically, any information regarding the value assumed by $Y$ does not affect the information regarding the value assumed by $X$
Implication:
so, independent random variables implies that
$$f_{X|Y}(x|y)=f_X(x)$$
similarly,
$$f_{Y|X}(y|x)=f_Y(y)$$
applying this in the Conditional distribution,
$$f_{XY}(x,y)=f_{X|Y}(x|y). f_Y(y)$$
$$f_{XY}(x,y)=f_X(x). f_Y(y)$$
similarly,
$$f_{XY}(x,y)=f_{Y|X}(y|x).f_X(x)$$
$$f_{XY}(x,y)=f_Y(y).f_X(x)$$
Therefore, two random variables $X$ and $Y$ with joint pdf $f_{XY}(x,y)$ and the marginal probability density functions $f_X(x)$ and $ f_Y(y)$ are independent if and only if $f_{XY}(x,y)=f_X(x).f_Y(y)$
Note: This is not a definition of independent random variables but can be a working rule to establish the independence of two random variables
Example 3:
If the joint probability density of two random variables is
$$f_{XY}(x, y) =\begin{cases} 6e^{-2x-3y} & x, y > 0 \\ 0 & \text{elsewhere}\end{cases}$$
Check whether $X$ and $Y$ are independent.
$$f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x, y)\, dy$$
$$= \int_{0}^{\infty} 6e^{-2x-3y}\, dy = 6e^{-2x} \left( \frac{e^{-3y}}{-3} \right)_0^{\infty} $$
$$= \begin{cases} 2e^{-2x} & x > 0 \\ 0 & \text{elsewhere} \end{cases}$$
$$ f_Y(y) = \int_{-\infty}^{\infty} f_{XY}(x, y)\, dx $$
$$ = \int_{0}^{\infty} 6e^{-2x-3y}\, dx = 6e^{-3y} \left( \frac{e^{-2x}}{-2} \right)_0^{\infty} $$
$$= \begin{cases} 3e^{-3y} & y > 0 \\ 0 & \text{elsewhere} \end{cases} $$
Now
$$ f_X(x) \cdot f_Y(y) = (2)e^{-2x} \cdot (3)e^{-3y} $$
$$= 6e^{-2x-3y} = f_{XY}(x, y)$$
$\therefore$ $X$ and $Y$ are independent.
Example 4:
Let the joint pdf of $(X, Y)$ is
$$f_{XY}(x, y) =\begin{cases} \frac{1}{4}(1 + xy) & |x| < 1;\ |y| < 1 \\ 0 & \text{elsewhere} \end{cases} $$
Find whether $X$ and $Y$ are not independent.
Marginal pdfs of $X$ and $Y$:
$$f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x, y)\, dy = \int_{-1}^{1} \frac{1}{4}(1 + xy)\, dy $$
$$= \frac{1}{4}\left( y + \frac{xy^2}{2} \right)_{-1}^{1} $$
$$ f_X(x) =\begin{cases} \frac{1}{2} & -1 < x < 1 \\ 0 & \text{elsewhere} \end{cases} $$
$$f_Y(y) = \int_{-\infty}^{\infty} f_{XY}(x, y)\, dx = \int_{-1}^{1} \frac{1}{4}(1 + xy)\, dx$$
$$= \frac{1}{4}\left( x + \frac{x^2 y}{2} \right)_{-1}^{1}$$
$$f_Y(y) = \begin{cases} \frac{1}{2} & -1 < y < 1 \\ 0 & \text{elsewhere} \end{cases} $$
Now $f_X(x) \cdot f_Y(y) \neq f_{XY}(x, y)$.
Hence $X$ and $Y$ are not independent.
Joint Distributions Function
We shall extend the concept of commulative distribution function (cdf) to the 2-variable case. We write $F_{XY}(x, y)$ as the probability that $X$ takes on a value less than or equal to $x$ (i.e., $X \leq x$) and $Y$ takes on a value less than or equal to $y$ (i.e., $Y \leq y$) and we refer to the corresponding function $F$ as the joint commulative function of the 2 random variables.
$$\therefore \quad F_{XY}(x, y) = p(X \leq x, Y \leq y)$$
In case of continuous case, $F_{XY}(x, y)$ can be formulated as,
$$F_{XY}(x, y) = \int_{-\infty}^{y} \int_{-\infty}^{x} f_{XY}(u, v) \, du dv$$
(we use $u$ and $v$ as dummy variables for integration purpose).
Marginal distribution functions
From the knowledge of joint distribution function $F_{XY}(x, y)$ it is possible to obtain the distribution function of $X$ and $Y$ which are said to be cdf of $X$ and $Y$ respectively.
i.e.,
$$F_X(x) = p(X \leq x) = p(X \leq x, Y < \infty)$$
$$= F_{XY}(x, \infty) = \lim_{y \to \infty} F_{XY}(x, y)$$
Similarly,
$$F_Y(y) = p(Y \leq y) = p(X < \infty, Y \leq y) = F_{XY}(\infty, y)$$
$$= \lim_{x \to \infty} F_{XY}(x, y)$$
Let us use this definition to the proceeding example.
$$F_X(x) = \lim_{y \to \infty} F_{XY}(x, y)$$
$$= \lim_{y \to \infty} (1 – e^{-2x})(1 – e^{-3y})$$
$$= 1 – e^{-2x}$$
and
$$F_Y(y) = \lim_{x \to \infty} F_{XY}(x, y)$$
$$= \lim_{x \to \infty} (1 – e^{-2x})(1 – e^{-3y})$$
$$= 1 – e^{-3y}$$
Properties of Joint Distributions
- $F_{XY}(-\infty, y) = 0$, (ii) $F_{XY}(x, -\infty) = 0$ (iii) $F_{XY}(-\infty, -\infty)$ (iv) $F_{XY}(\infty, \infty) = 1$.
- If the R.V is continuous then its j.p.d.f $f_{XY}(x, y)$ obtained from its joint p.d.f $F_{XY}(x, y)$ $$f_{XY}(x, y) = \frac{\partial^2 F}{\partial x \partial y} = \frac{\partial^2 F}{\partial y \partial x}$$
- For the real numbers $a_1, b_1, a_2$ and $p(a_1 < X \leq b_1, a_2 < Y \leq b_2) = F_{XY}(a_1a_2) + F_{XY}(b_1b_2) – F_{XY}(a_1b_2) – F_{XY}(b_1a_2)$.
- If $X$ and $Y$ are mutually independent, then $F_{XY}(x, y) = F_X(x) \cdot F_Y(y)$.