Chapter 9.5, Problem 66E

The likelihood function $L\left(y_1,y_2,\dots ,y_n|\theta \right)$ takes on different values depending on the arguments $\left(y_1,y_2,\dots ,y_n\right)$. A method for deriving a minimal sufficient statistic developed by Lehmann and Scheffé uses the ratio of the likelihoods evaluated at two points, $\left(x_1,x_2,\dots ,x_n\right)$ and $\left(y_1,y_2,\dots ,y_n\right)$:

$\frac{L\left(x_1,x_2,\dots ,x_n|\theta \right)}{L\left(y_1,y_2,\dots ,y_n|\theta \right)}.$

Many times it is possible to find a function $g\left(x_1,x_2,\dots ,x_n\right)$ such that this ratio is free of the unknown parameter θ if and only if $g\left(x_1,x_2,\dots ,x_n\right)=g\left(y_1,y_2,\dots ,y_n\right)$. If such a function g can be found, then $g\left(Y_1,Y_2,\dots ,Y_n\right)$ is a minimal sufficient statistic for θ.

1. a a Let $Y_1,Y_2,\dots ,Y_n$ be a random sample from a Bernoulli distribution (see Example 9.6 and Exercise 9.65) with p unknown.
   1. i Show that

$\frac{L\left(x_1,x_2,\dots ,x_n|\theta \right)}{L\left(y_1,y_2,\dots ,y_n|\theta \right)}={\left(\frac{p}{1-p}\right)}^{\sum x_i-\sum y_i}.$

1. ii Argue that for this ratio to be independent of p, we must have

${\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i}-{\displaystyle \underset{i=1}{\overset{n}{\sum }}{y}_i=0}\text{or}{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i}={\displaystyle \underset{i=1}{\overset{n}{\sum }}{y}_i}.$

1. iii Using the method of Lehmann and Scheffé, what is a minimal sufficient statistic for p? How does this sufficient statistic compare to the sufficient statistic derived in Example 9.6 by using the factorization criterion?
2. b Consider the Weibull density discussed in Example 9.7.
3. i Show that

$\frac{L\left(x_1,x_2,\dots ,x_n|\theta \right)}{L\left(y_1,y_2,\dots ,y_n|\theta \right)}=\left(\frac{x_1{x}_2\cdots x_n}{y_1{y}_2\cdots y_n}\right)\exp \left[-\frac{1}{\theta }\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i^2}-{\displaystyle \underset{i=1}{\overset{n}{\sum }}{y}_i^2}\right)\right].$

1. ii Argue that ${\sum }_{i=1}^n{Y}_i^2$ is a minimal sufficient statistic for θ.

EXAMPLE 9.6

Let $Y_1,Y_2,\dots ,Y_n$ denote a random sample from a distribution where $P\left(Y_i=1\right)=p$ and $P\left(Y_i=0\right)=1-p$, with p unknown (such random variables are often called Bernoulli variables). Use the factorization criterion to find a sufficient statistic that best summarizes the data. Give an MVUE for p.

*9.65 In this exercise, we illustrate the direct use of the Rao–Blackwell theorem. Let $Y_1,Y_2,\dots ,Y_n$ be independent Bernoulli random variables with

$p\left(y_i|p\right)=p^{y_i}{\left(1-p\right)}^{1-y_i},y_i=0,1.$

That is, $P\left(Y_i=1\right)=p$ and $P\left(Y_i=0\right)=1-p$. Find the MVUE of p(l – p), which is a term in the variance of Y_i or $W={\sum }_{i=1}^n{Y}_i$, by the following steps.

1. a Let

$T=\{\begin{array}{ll}1,\hfill & \text{if}{Y}_1=1\text{and}{Y}_2=0,\hfill \\ 0,\hfill & \text{otherwise}\text{.}\hfill \end{array}$

Show that E(T) = p(1 – p).

1. b Show that

$P\left(T=1|W=w\right)=\frac{w\left(n-w\right)}{n\left(n-1\right)}.$

1. c Show that

$E\left(T|W\right)=\frac{n}{n-1}\left[\frac{W}{n}\left(1-\frac{W}{n}\right)\right]=\frac{n}{n-1}\bar{Y}\left(1-\bar{Y}\right)$

and hence that $n\bar{Y}\left(1-\bar{Y}\right)/\left(n-1\right)$ is the MVUE of p(1 – p).

EXAMPLE 9.7

Suppose that $Y_1,Y_2,\dots ,Y_n$ denote a random sample from the Weibull density function, given by

$f\left(y|\theta \right)=\{\begin{array}{ll}\left(\frac{2y}{\theta }\right)e^{-y^2/\theta },\hfill & y>0\hfill \\ 0,\hfill & \text{elsewhere}\text{.}\hfill \end{array}$

Find an MVUK for θ.

To determine

(i). Prove that $\frac{L\left(x_1,x_2,\cdots ,x_n|p\right)}{L\left(y_1,y_2,\cdots ,y_n|p\right)}={\left(\frac{p}{1-p}\right)}^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}$.

(ii). Prove that the ratio in subpart (i) to be independent of p; it must be ${\displaystyle \sum x_i}-{\displaystyle \sum y_i}=0$ or ${\displaystyle \sum x_i}={\displaystyle \sum y_i}$.

(iii). Find the minimal sufficient statistic for p using the method of Lehmann and Scheffe. Explain how this sufficient statistic is compared to the sufficient statistic derived in example 9.6 using the factorization theorem.

Answer to Problem 66E

(i). It is proved that $\frac{L\left(x_1,x_2,\cdots ,x_n|p\right)}{L\left(y_1,y_2,\cdots ,y_n|p\right)}={\left(\frac{p}{1-p}\right)}^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}$.

(ii). It is proved that the ratio in subpart (i) to be independent of p; it must be ${\displaystyle \sum x_i}-{\displaystyle \sum y_i}=0\text{ or }{\displaystyle \sum x_i}={\displaystyle \sum y_i}$.

(iii). The minimal sufficient statistic for p is ${\displaystyle \sum _{i=1}^n{Y}_i}$. This is the same as the sufficient statistic derived in example 9.6 using the factorization theorem.

Explanation of Solution

(i).

Given that $Y_1,Y_2,\mathrm{...},Y_n$ be independent Bernoulli random variables, the probability mass function is given as follows:

$P\left(y_i|p\right)=p^{y_i}{\left(1-p\right)}^{1-y_i}\text{ ; }{y}_i=0,1$.

The likelihood function of $y_i$ is given as follows:

$\begin{array}{c}L\left(y_1,y_2,\cdots ,y_n|p\right)={\displaystyle \prod _{i=1}^n{p}^{y_i}{\left(1-p\right)}^{1-y_i}}\\ =p^{{\displaystyle \sum y_i}}{\left(1-p\right)}^{{\displaystyle \sum \left(1-y_i\right)}}\\ =p^{{\displaystyle \sum y_i}}{\left(1-p\right)}^{n-{\displaystyle \sum y_i}}\end{array}$

Similarly, likelihood function of $x_i$ is given as follows:

$L\left(x_1,x_2,\cdots ,x_n|p\right)=p^{{\displaystyle \sum x_i}}{\left(1-p\right)}^{n-{\displaystyle \sum x_i}}$.

The ratio of the two likelihood functions can be written as follows:

$\begin{array}{c}\frac{L\left(x_1,x_2,\cdots ,x_n|p\right)}{L\left(y_1,y_2,\cdots ,y_n|p\right)}=\frac{p^{{\displaystyle \sum x_i}}{\left(1-p\right)}^{n-{\displaystyle \sum x_i}}}{p^{{\displaystyle \sum y_i}}{\left(1-p\right)}^{n-{\displaystyle \sum y_i}}}\\ =\frac{p^{{\displaystyle \sum x_i}}{p}^{-{\displaystyle \sum y_i}}}{{\left(1-p\right)}^{n-{\displaystyle \sum y_i}}{\left(1-p\right)}^{-\left(n-{\displaystyle \sum x_i}\right)}}\\ =\frac{p^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}}{{\left(1-p\right)}^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}}\\ ={\left(\frac{p}{1-p}\right)}^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}\end{array}$

Thus, it is proved that $\frac{L\left(x_1,x_2,\cdots ,x_n|p\right)}{L\left(y_1,y_2,\cdots ,y_n|p\right)}={\left(\frac{p}{1-p}\right)}^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}$.

(ii).

Form subpart (i), the ratio of the two likelihood functions is given as follows:

$\frac{L\left(x_1,x_2,\cdots ,x_n|p\right)}{L\left(y_1,y_2,\cdots ,y_n|p\right)}={\left(\frac{p}{1-p}\right)}^{{\displaystyle \sum x_i}-{\displaystyle \sum y_i}}$

The above ratio will be independent of p if and only if the term in the power is equal to 0.

That is,

${\displaystyle \sum x_i}-{\displaystyle \sum y_i}=0\text{ or }{\displaystyle \sum x_i}={\displaystyle \sum y_i}$.

Thus, it is proved that the ratio in subpart (i) to be independent of p; it must be ${\displaystyle \sum x_i}-{\displaystyle \sum y_i}=0\text{ or }{\displaystyle \sum x_i}={\displaystyle \sum y_i}$.

(iii).

Form subpart (ii), the ratio in subpart (i) to be independent of p; it must be ${\displaystyle \sum x_i}={\displaystyle \sum y_i}$.

By the method of Lehmann–Scheffe, $g\left(Y_1,Y_2,\cdots ,Y_n\right)={\displaystyle \sum _{i=1}^n{Y}_i}$.

Hence, it shows that the minimal sufficient statistic for p is ${\displaystyle \sum _{i=1}^n{Y}_i}$. This is the same as the sufficient statistic derived in example 9.6 using the factorization theorem.

To determine

(i). Prove that $\frac{L\left(x_1,x_2,\cdots ,x_n|\theta \right)}{L\left(y_1,y_2,\cdots ,y_n|\theta \right)}=\frac{\left(x_1{x}_2\cdots x_n\right)}{\left(y_1{y}_2\cdots y_n\right)}\exp \left[-\frac{1}{\theta }\left({\displaystyle \sum _{i=1}^n{x}_i^2}-{\displaystyle \sum _{i=1}^n{y}_i^2}\right)\right]$.

(ii). Prove that ${\displaystyle \sum _{i=1}^n{Y}_i^2}$ is a minimal sufficient statistic for θ.

Answer to Problem 66E

(i). It is proved that $\frac{L\left(x_1,x_2,\cdots ,x_n|\theta \right)}{L\left(y_1,y_2,\cdots ,y_n|\theta \right)}=\frac{\left(x_1{x}_2\cdots x_n\right)}{\left(y_1{y}_2\cdots y_n\right)}\exp \left[-\frac{1}{\theta }\left({\displaystyle \sum _{i=1}^n{x}_i^2}-{\displaystyle \sum _{i=1}^n{y}_i^2}\right)\right]$.

(ii). It is proved that ${\displaystyle \sum _{i=1}^n{Y}_i^2}$ is a minimal sufficient statistic for θ.

Explanation of Solution

(i).

Given that $Y_1,Y_2,\mathrm{...},Y_n$ be a random sample from the Weibull density function, the probability density function is given as follows:

$\left(y|\theta \right)=\{\begin{array}{l}\left(\frac{2y}{\theta }\right)e^{\frac{-y^2}{\theta }}\text{ ; }{y}_i>0,\\ 0\text{ ; Elsewhere}\text{.}\end{array}$.

The likelihood function of $y_i$ is given as follows:

$\begin{array}{c}L\left(y_1,y_2,\cdots ,y_n|\theta \right)={\displaystyle \prod _{i=1}^n\left[\left(\frac{2y_i}{\theta }\right)\exp \left(\frac{-y_i^2}{\theta }\right)\right]}\\ ={\left(\frac{2}{\theta }\right)}^n\left({\displaystyle \prod _{i=1}^n{y}_i}\right)\exp \left(\frac{-{\displaystyle \sum _{i=1}^n{y}_i^2}}{\theta }\right)\end{array}$

Similarly, the likelihood function of $x_i$ is given as follows:

$L\left(x_1,x_2,\cdots ,x_n|\theta \right)={\left(\frac{2}{\theta }\right)}^n\left({\displaystyle \prod _{i=1}^n{x}_i}\right)\exp \left(\frac{-{\displaystyle \sum _{i=1}^n{x}_i^2}}{\theta }\right)$.

The ratio of the two likelihood functions can be written as follows:

$\begin{array}{c}\frac{L\left(x_1,x_2,\cdots ,x_n|\theta \right)}{L\left(y_1,y_2,\cdots ,y_n|\theta \right)}=\frac{{\left(\frac{2}{\theta }\right)}^n\left({\displaystyle \prod _{i=1}^n{x}_i}\right)\exp \left(\frac{-{\displaystyle \sum _{i=1}^n{x}_i^2}}{\theta }\right)}{{\left(\frac{2}{\theta }\right)}^n\left({\displaystyle \prod _{i=1}^n{y}_i}\right)\exp \left(\frac{-{\displaystyle \sum _{i=1}^n{y}_i^2}}{\theta }\right)}\\ =\frac{\left(x_1{x}_2\cdots x_n\right)}{\left(y_1{y}_2\cdots y_n\right)}\exp \left[-\frac{1}{\theta }\left({\displaystyle \sum _{i=1}^n{x}_i^2}-{\displaystyle \sum _{i=1}^n{y}_i^2}\right)\right]\end{array}$

Thus, it is proved that $\frac{L\left(x_1,x_2,\cdots ,x_n|\theta \right)}{L\left(y_1,y_2,\cdots ,y_n|\theta \right)}=\frac{\left(x_1{x}_2\cdots x_n\right)}{\left(y_1{y}_2\cdots y_n\right)}\exp \left[-\frac{1}{\theta }\left({\displaystyle \sum _{i=1}^n{x}_i^2}-{\displaystyle \sum _{i=1}^n{y}_i^2}\right)\right]$.

(ii).

Form subpart (i), the ratio of the two likelihood functions is given as follows:

$\frac{L\left(x_1,x_2,\cdots ,x_n|\theta \right)}{L\left(y_1,y_2,\cdots ,y_n|\theta \right)}=\frac{\left(x_1{x}_2\cdots x_n\right)}{\left(y_1{y}_2\cdots y_n\right)}\exp \left[-\frac{1}{\theta }\left({\displaystyle \sum _{i=1}^n{x}_i^2}-{\displaystyle \sum _{i=1}^n{y}_i^2}\right)\right]$

The above ratio will be independent of θ if and only if the term in the power of exponential term is equal to 0.

That is,

${\displaystyle \sum _{i=1}^n{x}_i^2}-{\displaystyle \sum _{i=1}^n{y}_i^2}=0\text{ or }{\displaystyle \sum _{i=1}^n{x}_i^2}={\displaystyle \sum _{i=1}^n{y}_i^2}$.

By the method of Lehmann-Scheffe, $g\left(Y_1,Y_2,\cdots ,Y_n\right)={\displaystyle \sum _{i=1}^n{Y}_i^2}$.

Hence, it shows that the minimal sufficient statistic for θ is ${\displaystyle \sum _{i=1}^n{Y}_i^2}$.