# Statistical Inference

### Ph.D. program in Engineering and Applied Sciences

Università degli Studi di Bergamo, Italy

A.Y. 2023/2024

Extended syllabus

Statistical inference terminology: population and sample, parameters and estimators, distributions and random variables (r.v.). Refer to Chapter 6 of Mood, Graybill, and Boes (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-to-inference.html 2 Refer to Chapter 7 of Mood et al. (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-toinference.html

Estimators and sampling distributions Refer to Chapter 7 of Mood et al. (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-toinference.html

Estimators of the parameters for common r.v. (e.g., Gaussian, Bernoulli, Poisson, Gamma, etc.)

Properties of estimators (unbiasedness, efficiency, Cramér-Rao inequality, etc.).

Method of moments estimators (MME) Refer to Chapter 7 (Section 2.2) of Mood et al. (1974) or Section 3 at https://bookdown.org/probability/inference/maximum-likelihood.html

Maximum likelihood estimators (MLE) Refer to Chapter 7 (Section 2.2) of Mood et al. (1974) or Section 4 at https://bookdown.org/probability/inference/maximum-likelihood.html

Likelihood function and its properties;

Examples of exact (closed form) solutions for the MLE;

Numerical algorithms for MLE: Nelder-Mead, Newton-Raphson and Quasi-Newton methods (e.g., BFGS and L-BFGS-B) Refer to Chapter 3 and 4 of Everitt (2012) and Chapter 4 of Rustagi (2014)

Hypothesis testing (HT) Refer to Chapter 9 of Mood et al. (1974) or Sections 6 and 7 at https://bookdown.org/probability/_inference2/hypothesis-testing.html

HT based on pivotal quantities:

HT based on the likelihood: Likelihood ratio test, Wald test statistics, Score statistics

Type I, Type II errors and power in HT (Neyman and Pearson Lemma)

Confidence intervals (CI) Refer to Chapter 8 of Mood et al. (1974) or Section 6 https://bookdown.org/probability/statistics/confidenceinterval.html

CI based on pivotal quantities

CI based on hypothesis tests (Wald’s CI and LRT CI)

Introduction to linear models and regression Chapters 3, 4 and 5 of Seber and Lee (2012)

Linear models in matrix form and assumptions

Ordinary Least Squares estimator (OLSE) and Gauss-Markov theorem

Equivalence between OLSE and MLE for linear models and inference for model’s parameters

Violation (and remedies) of OLS assumptions

In addition to the theoretical concepts, for the most relevant topics will be provided computational examples using the R programming language.

Essential bibliography

Everitt, B. (2012). Introduction to optimization methods and their application in statistics.

Mood, A., Graybill, F., & Boes, D. (1974). Introduction to statistical theory. In: McGraw-Hill, New York. Rustagi, J. S. (2014). Optimization techniques in statistics: Elsevier.

Seber, G. A., & Lee, A. J. (2012). Linear regression analysis (Vol. 329): John Wiley & Sons.