1 / 37

STATISTICAL INFERENCE PART II SOME PROPERTIES OF ESTIMATORS

STATISTICAL INFERENCE PART II SOME PROPERTIES OF ESTIMATORS. SOME PROPERTIES OF ESTIMATORS. θ: a parameter of interest; unknown Previously, we found good(?) estimator(s) for θ or its function g(θ). Goal: Check how good are these estimator(s). Or are they good at all?

elke
Download Presentation

STATISTICAL INFERENCE PART II SOME PROPERTIES OF ESTIMATORS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STATISTICAL INFERENCEPART IISOME PROPERTIES OF ESTIMATORS

  2. SOME PROPERTIES OF ESTIMATORS • θ: a parameter of interest; unknown • Previously, we found good(?) estimator(s) for θ or its function g(θ). • Goal: • Check how good are these estimator(s). Or are they good at all? • If more than one good estimator is available, which one is better?

  3. Bias of for estimating  If is UE of , SOME PROPERTIES OF ESTIMATORS • UNBIASED ESTIMATOR (UE): An estimator is an UE of the unknown parameter , if Otherwise, it is a BiasedEstimator of .

  4. SOME PROPERTIES OF ESTIMATORS • ASYMPTOTICALLY UNBIASED ESTIMATOR (AUE): An estimator is an AUE of the unknown parameter , if

  5. SOME PROPERTIES OF ESTIMATORS • CONSISTENT ESTIMATOR (CE): An estimator which converges in probability to an unknown parameter  for all  is called a CE of . For large n, a CE tends to be closer to the unknown population parameter. • MLEs are generally CEs.

  6. EXAMPLE For a r.s. of size n, By WLLN,

  7. If is smaller, is a better estimator of . MEAN SQUARED ERROR (MSE) • The Mean Square Error (MSE) of an estimator for estimating  is

  8. MEAN SQUARED ERROR CONSISTENCY • Tn is called mean squared error consistent (or consistent in quadratic mean) if E{Tn}20 as n. Theorem: Tnis consistent in MSE iff i)Var(Tn)0 as n. • If E{Tn}20 as n, Tnis also a CE of .

  9. EXAMPLES X~Exp(), >0. For a r.s of size n, consider the following estimators of , and discuss their bias and consistency. Which estimator is better?

  10. SUFFICIENT STATISTICS • X, f(x;),  • X1, X2,…,Xn • Y=U(X1, X2,…,Xn ) is a statistic. • A sufficient statistic, Y, is a statistic which contains all the information for the estimation of .

  11. SUFFICIENT STATISTICS • Given the value of Y, the sample contains no further information for the estimation of . • Y is a sufficient statistic (ss) for  if the conditional distribution h(x1,x2,…,xn|y) does not depend on  for every given Y=y. • A ss for is not unique: • If Y is a ss for , then any 1-1 transformation of Y, say Y1=fn(Y) is also a ss for .

  12. SUFFICIENT STATISTICS • The conditional distribution of sample rvs given the value of y of Y, is defined as • If Y is a ss for , then Not depend on  for every given y. ss for  may include y or constant. • Also, the conditional range of Xi given y not depend on .

  13. SUFFICIENT STATISTICS EXAMPLE: X~Ber(p). For a r.s. of size n, show that is a ss for p.

  14. SUFFICIENT STATISTICS • Neyman’s Factorization Theorem:Y is a ss for  iff The likelihood function Does not depend on xi except through y Not depend on  (also in the range of xi.) where k1and k2 are non-negative functions.

  15. EXAMPLES 1. X~Ber(p). For a r.s. of size n, find a ss for p if exists.

  16. EXAMPLES 2. X~Beta(θ,2). For a r.s. of size n, find a ss for θ.

  17. SUFFICIENT STATISTICS • A ss may not exist. • Jointly ss Y1,Y2,…,Yk may be needed. Example: Example 10.2.5 in Bain and Engelhardt (page 342 in 2nd edition), X(1) and X(n) are jointly ss for  • If the MLE of  exists and unique and if a ss for  exists, then MLE is a function of a ss for .

  18. EXAMPLE X~N(,2). For a r.s. of size n, find jss for  and 2.

  19. MINIMAL SUFFICIENT STATISTICS • If is a ss for θ, then, is also a ss for θ. But, the first one does a better job in data reduction. A minimalss achieves the greatest possible reduction.

  20. MINIMAL SUFFICIENT STATISTICS • A ss T(X) is called minimal ss if, for any other ss T’(X),T(x) is a function of T’(x). • THEOREM: Let f(x;) be the pmf or pdf of a sampleX1, X2,…,Xn. Suppose there exist a function T(x) such that, for two sample points x1,x2,…,xn and y1,y2,…,yn, the ratio is constant with respect to  iff T(x)=T(y). Then, T(X) is a minimal sufficient statistic for .

  21. EXAMPLE • X~N(,2) where 2 is known. For a r.s. of size n, find minimal ss for . Note: A minimal ss is also not unique. Any 1-to-1 function is also a minimal ss.

  22. RAO-BLACKWELL THEOREM • Let X1, X2,…,Xn have joint pdf or pmf f(x1,x2,…,xn;) and let S=(S1,S2,…,Sk) be a vector of jss for . If T is an UE of () and (T)=E(TS), then • (T) is an UE of() . • (T) is a fn of S, so it is also jss for . • Var((T) ) Var(T) for all . • (T) is a uniformly better unbiased estimator of () .

  23. RAO-BLACKWELL THEOREM • Notes: • (T)=E(TS) is at least as good as T. • For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs.

  24. Example • Hogg & Craig, Exercise 10.10 • X1,X2~Exp(θ) • Find joint p.d.f. of ss Y1=X1+X2 for θ and Y2=X2. • Show that Y2 is UE of θ with variance θ². • Find φ(y1)=E(Y2|Y1) and variance of φ(Y1).

  25. ANCILLARY STATISTIC • A statistic S(X) whose distribution does not depend on the parameter  is called an ancillary statistic. • Unlike a ss, an ancillary statistic contains no information about .

  26. Example • Example 6.1.8 in Casella & Berger, page 257: Let Xi~Unif(θ,θ+1) for i=1,2,…,n Then, range R=X(n)-X(1) is an ancillary statistic because its pdf does not depend on θ.

  27. COMPLETENESS • Let {f(x; ), } be a family of pdfs (or pmfs) and U(x) be an arbitrary function of x not depending on . If requires that the function itself equal to 0 for all possible values of x; then we say that this family is a complete family of pdfs (or pmfs). i.e., the only unbiased estimator of 0 is 0 itself.

  28. EXAMPLES 1. Show that the family {Bin(n=2,); 0<<1} is complete.

  29. EXAMPLES 2. X~Uniform(,). Show that the family {f(x;), >0}is not complete.

  30. COMPLETE AND SUFFICIENT STATISTICS (css) • Y is a complete and sufficient statistic (css) for  if Y is a ss for  and the family is complete. The pdf of Y. 1) Y is a ss for . 2) u(Y) is an arbitrary function of Y. E(u(Y))=0 for all  implies that u(y)=0 for all possible Y=y.

  31. (n-1)S2/ 2 ~ By Basu theorem, and S2are independent. BASU THEOREM • If T(X) is a complete and minimal sufficient statistic, then T(X) is independent of every ancillary statistic. • Example:X~N(,2). S2 Ancillary statistic for 

  32. BASU THEOREM • Example: • Let T=X1+ X2 and U=X1 - X2 • We know that T is a complete minimal ss. • U~N(0, 2)  distribution free of   T and U are independent by Basu’s Theorem X1, X2~N(,2), independent, 2 known.

  33. THE MINIMUM VARIANCE UNBIASED ESTIMATOR • Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then (T)=E(TS)is • an UE of , i.e.,E[(T)]=E[E(TS)]= and • the MVUE of .

  34. LEHMANN-SCHEFFE THEOREM • Let Y be a cssfor . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . • Y css for . • T(y)=fn(y) and E[T(Y)]=. • T(Y) is the UMVUE of . • So, it is the best estimator of .

  35. THE MINIMUM VARIANCE UNBIASED ESTIMATOR • Let Y be a cssfor . Since Y is complete, there could be only a unique function of Y which is an UE of . • Let U1(Y) and U2(Y) be two function of Y. Since they are UE’s, E(U1(Y)U2(Y))=0 imply W(Y)=U1(Y)U2(Y)=0 for all possible values of Y. Therefore, U1(Y)=U2(Y) for all Y.

  36. Example • Let X1,X2,…,Xn ~Poi(μ). Find UMVUE of μ. • Solution steps: • Show that is css for μ. • Find a statistics (such as S*) that is UE of μ and a function of S. • Then, S* is UMVUE of μ by Lehmann-Scheffe Thm.

  37. Note • The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique.

More Related