Statistical Inference

Link to materials

Extended syllabus

1. 1. Statistical inference terminology: population and sample, parameters and estimators, distributions and random variables (r.v.). Refer to Chapter 6 of Mood, Graybill, and Boes (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-to-inference.html 2 Refer to Chapter 7 of Mood et al. (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-toinference.html

   2. Estimators and sampling distributions Refer to Chapter 7 of Mood et al. (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-toinference.html

      1. Estimators of the parameters for common r.v. (e.g., Gaussian, Bernoulli, Poisson, Gamma, etc.)

      2. Properties of estimators (unbiasedness, efficiency, Cramér-Rao inequality, etc.).

   3. Method of moments estimators (MME) Refer to Chapter 7 (Section 2.2) of Mood et al. (1974) or Section 3 at https://bookdown.org/probability/inference/maximum-likelihood.html

   4. Maximum likelihood estimators (MLE) Refer to Chapter 7 (Section 2.2) of Mood et al. (1974) or Section 4 at https://bookdown.org/probability/inference/maximum-likelihood.html

      1. Likelihood function and its properties;

      2. Examples of exact (closed form) solutions for the MLE;

      3. Numerical algorithms for MLE: Nelder-Mead, Newton-Raphson and Quasi-Newton methods (e.g., BFGS and L-BFGS-B) Refer to Chapter 3 and 4 of Everitt (2012) and Chapter 4 of Rustagi (2014)

   5. Hypothesis testing (HT) Refer to Chapter 9 of Mood et al. (1974) or Sections 6 and 7 at https://bookdown.org/probability/_inference2/hypothesis-testing.html

      1. HT based on pivotal quantities:

      2. HT based on the likelihood: Likelihood ratio test, Wald test statistics, Score statistics

      3. Type I, Type II errors and power in HT (Neyman and Pearson Lemma)

   6. Confidence intervals (CI) Refer to Chapter 8 of Mood et al. (1974) or Section 6 https://bookdown.org/probability/statistics/confidenceinterval.html

      1. CI based on pivotal quantities

      2. CI based on hypothesis tests (Wald’s CI and LRT CI)

   7. Introduction to linear models and regression Chapters 3, 4 and 5 of Seber and Lee (2012)

      1. Linear models in matrix form and assumptions

      2. Ordinary Least Squares estimator (OLSE) and Gauss-Markov theorem

      3. Equivalence between OLSE and MLE for linear models and inference for model’s parameters

      4. Violation (and remedies) of OLS assumptions

In addition to the theoretical concepts, for the most relevant topics will be provided computational examples using the R programming language.

Essential bibliography

• Everitt, B. (2012). Introduction to optimization methods and their application in statistics.

• Mood, A., Graybill, F., & Boes, D. (1974). Introduction to statistical theory. In: McGraw-Hill, New York. Rustagi, J. S. (2014). Optimization techniques in statistics: Elsevier.

• Seber, G. A., & Lee, A. J. (2012). Linear regression analysis (Vol. 329): John Wiley & Sons.