Introduction
This notes confines to MOM in obtaining Point Estimators for parameters
Suggested Reading: [CABE] Casella, G., & Berger, R. L. (2002). Statistical inference (Vol. 2). Pacific Grove, CA: Duxbury; specifically, Chapters 6 and 7
Keywords:
- Estimator
- Estimate
- Likelihood function
- Method of Moments (MOM)
- Maximum Likelihood Estimates (MLE)
- Bias, Variance and Mean Squared Error (MSE) of an estimator
Objectives
- Method of finding estimators
- Criteria to find a “best” estimator
- Assessing tools – goodness of estimator
An estimator of $\tau(\boldsymbol\theta)$, a function of parameter is any function $W(X_1,X_2,\cdots,X_n)$ of a sample; that is any statistic is a point estimator
Here, $\boldsymbol{\theta}=(\theta_1, \theta_2,\cdots,\theta_k)$
Method of Moments:
Let X be a random variable ~ $f(X~|~\theta)$ where
$\boldsymbol\theta=(\theta_1, \theta_2,\cdots,\theta_k)$ where K is the number of parameters
Let $\mu’_r = E[X^{r}]$
$\mu’_r$ will be a function of $\theta$
Let $X_1, X_2,\cdots,X_n$ be a random sample from $f(X~|~\theta)$
Now, $m’_r=\frac{1}{n}\sum X_i^r$
Solving “k” equations arising from equating these two moments
$\mu’_1=m’_1$
$\mu’_2=m’_2$
$\vdots$
$\mu’_k=m’_k$
Therefore estimators are obtained by comparing population moments and sample moments
Examples of MOM:
Example 1.
Let $X_1, X_2,\cdots,X_n$ be a random sample from $N(\mu,\sigma^2)$
$\theta_1 = \mu$
$\theta_2=\sigma^2$
$\mu’_1=E(X)=\mu$
$\mu’_2=E(X^2)=\sigma^2+\mu^2$
$\Rightarrow\frac{\sum X_i}{n}=\mu$
$\frac{\sum{X_i}^2}{n}=\sigma^2+\mu^2$
$\hat\mu_{MM}=\overline {X}$
$\sigma^2=\frac{\sum{X_i}^2}{n}-\overline {X}^2$
$\Rightarrow \widehat{\sigma^2}_{MM} = \frac{\sum(X_i-\overline {X})^2}{n}$
Example 2.
Let $X_1, X_2,\cdots,X_n$ be a random sample from $f(x~|~\theta)=\theta e^{-\theta x}~~~~~~ x~>~0$
$E(X)=\frac{1}{\theta}$
$\mu’_1=\frac{\sum X_i}{n}=\overline {X}$
$\Rightarrow\frac{1}{\theta}=\overline {X}$
$\hat\theta_{MM}=\frac{1}{\overline {X}}$
Example 3.
Let $X_1, X_2,\cdots,X_n \sim \text{Uniform}(a,b)$
$\mu’_1=\frac{a+b}{2}$
$\mu’_2=\sigma^2+\mu^2$
$\mu’_2 = \frac{{(b-a)}^2}{12}+\mu^2$
$m’_1=\overline {X}=\mu$
$\frac{a+b}{2}=\overline {X}$
$\Rightarrow a+b=2\overline {X}~~~~~~~~~~(1)$
$\mu’_2=\frac{{(b-a)}^2}{12}+\overline {X}^2$
$\frac{{(b-a)}^2}{12}=\mu’_2-\overline {X}^2$
$=\frac{1}{n} \sum X_i^2-(\overline {X})^2$
$\Rightarrow b-a =2\sqrt 3 \sqrt{\frac{1}{n}\sum X_i^2-(\overline {X})^2}$
$=2\sqrt{3}\sqrt{\frac{1}{n}\sum(X_i-\overline {X})^2}~~~~~~~~~~(2)$
$(1)~\&~(2) ~ \Rightarrow$
$2b=2[\bar X+\sqrt3 \sqrt{\frac{1}{n}\sum (X_i-\overline {X})^2}]$
$b=\bar X+\sqrt3 \sqrt{\frac{1}{n}\sum (X_i-\overline {X})^2}$
$a=\bar X-\sqrt3 \sqrt{\frac{1}{n}\sum (X_i-\overline {X})^2}]$
Example 4.
Let $X_1, X_2,\cdots,X_n \sim \text{Binomial}(m,\theta)$
$\mu’_1=E(X)=m\theta$
$m’_1=\overline {X}$
$\mu’_2 = E(X^2) = m\theta(1-\theta) + m^2\theta^2$
$m’_2=\frac{\sum X_i^2}{n}$
$\Rightarrow\overline {X}=m\theta$
$\hat\theta_{MM} =\frac{\overline {X}}{m}$
Now let us estmate m so that $\hat\theta_{MM}$ will be completely obtained
$\frac{\sum X_i^2}{m}=m\theta(1-\theta)+m^2\theta^2$
$=m\frac{\overline {X}}{m}(1-\frac{\bar X}{m})+m^2\frac{(\overline {X})^2}{m^2}$
$=\overline {X}(1-\frac{\overline {X}}{m}) + \bar X^2$
$=\overline {X} – \frac{(\overline {X})^2}{m}+ \bar X^2$
$\frac{\sum{X_i}^2}{m}-\overline {X}^2=\overline {X}-\frac{\overline {X}^2}{m}$
$\frac{\overline {X}^2}{m}=\overline {X} – \frac{1}{n}\sum(X_i-\overline {X})^2$
$\Rightarrow \hat m_{MM} =\frac{\overline {X}^2}{\overline {X} – \frac{1}{n}\sum(X_i-\overline {X})^2}$
Example 5.
$X_1,\cdots,X_n \sim \text{Beta}(\alpha,~\beta)$
${M_1}^2=\mu’_1=\frac{\alpha}{\alpha+\beta}$
${M_2}^2=\mu’_2=\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}+\Big[\frac{\alpha}{\alpha+\beta}\Big]^2$
$=\frac{\alpha}{\alpha+\beta}~\frac{\beta}{\alpha+\beta}~\frac{1}{\alpha+\beta+1}+{M_1}^2$
$\alpha+\beta=\frac{\alpha}{M_1}$
$\alpha+\beta+1=\frac{\alpha+M_1}{M_1}$
$\frac{1}{\alpha+\beta+1}=\frac{M_1}{\alpha+M_1}$
$\Rightarrow M_2=M_1(1-M_1)\frac{M_1}{\alpha+M_1}+M_1^2$
$M_2-M_1^2=\frac{M_1^2(1-M_1)}{\alpha+M_1}$
$\alpha+M_1 =\frac{M_1^2(1-M_1)}{M_2-M_1^2}$
$\hat\alpha_{MM} =\frac{M_1^2(1-M_1)}{M_2-M_1^2}-M_1$
$\hat\beta_{MM}=\frac{\hat\alpha_{MM}}{M_1}-\hat\alpha_{MM}$
Example 6.
$X_1,\cdots,X_n\sim \text{Gamma}(\alpha,\beta)$
Then $M_1= E(X)=\alpha\beta$ and using $V(X)=\alpha\beta^2$ we get
$M_2=E(X^2)=\alpha\beta^2+\alpha^2\beta^2$
$=M_1\beta+M_1^2$
$\Rightarrow\hat\beta_{MM}=\frac{M_2-M_1^2}{M_1}$
$\Rightarrow\hat\alpha_{MM}=M_1\frac{1}{\hat\beta_{MM}}$
$=\frac{M_1^2}{M_2-M_1^2}$