# Statistical Inference

### Ph.D. program in Engineering and Applied Sciences

Università degli Studi di Bergamo, Italy

A.Y. 2023/2024

Syllabus

Statistical inference terminology: population and sample, parameters and estimators, distributions and random variables (r.v.)1

Estimators and sampling distributions2: a. Estimators of the parameters for common r.v. (e.g., Gaussian, Bernoulli, Poisson, Gamma, etc.) b. Properties of estimators (unbiasedness, efficiency, Cramér-Rao inequality, etc.)

Method of moments estimators (MME)

Maximum likelihood estimators (MLE): Likelihood function and its properties; Examples of exact (closed form) solutions for the MLE; Numerical algorithms for MLE: Nelder-Mead, Newton-Raphson and Quasi-Newton methods (e.g., BFGS and L-BFGS-B)

1 Refer to Chapter 6 of Mood, Graybill, and Boes (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-to-inference.html 2 Refer to Chapter 7 of Mood et al. (1974) or Section 2 at https://bookdown.org/probability/inference/introduction-toinference.html

3 Refer to Chapter 7 (Section 2.1) of Mood et al. (1974) or Section 3 at https://bookdown.org/probability/inference/method-of-moments.html

4 Refer to Chapter 7 (Section 2.2) of Mood et al. (1974) or Section 4 at https://bookdown.org/probability/inference/maximum-likelihood.html

5 Refer to Chapter 3 and 4 of Everitt (2012) and Chapter 4 of Rustagi (2014)

5. Hypothesis testing (HT)6

a. HT based on pivotal quantities:

b. HT based on the likelihood: Likelihood ratio test, Wald test statistics,

Score statistics

c. Type I, Type II errors and power in HT (Neyman and Pearson Lemma)

6. Confidence intervals (CI)7

a. CI based on pivotal quantities

b. CI based on hypothesis tests (Wald’s CI and LRT CI)

7. Introduction to linear models and regression8

a. Linear models in matrix form and assumptions

b. Ordinary Least Squares estimator (OLSE) and Gauss-Markov theorem

c. Equivalence between OLSE and MLE for linear models and inference for

model’s parameters

d. Violation (and remedies) of OLS assumptions

In addition to the theoretical concepts, for the most relevant topics will be provided computational examples using the R programming language.

Essential bibliography

Everitt, B. (2012). Introduction to optimization methods and their application in statistics.

Mood, A., Graybill, F., & Boes, D. (1974). Introduction to statistical theory. In: McGraw-Hill, New York. Rustagi, J. S. (2014). Optimization techniques in statistics: Elsevier.

Seber, G. A., & Lee, A. J. (2012). Linear regression analysis (Vol. 329): John Wiley & Sons.