Page 10

viii

CONTENTS

2.4

2.5

2.3.2

Schwarz’s heuristics . . . . . . . . . . . . . . . . . . . .

37

2.3.3

A first model selection theorem for linear models . . . .

38

. . . . . . . . . . .

43

2.4.1

Adaptive estimation in the minimax sense Minimax lower bounds

. . . . . . . . . . . . . . . . . .

45

2.4.2

Adaptive properties of penalized estimators for Gaussian sequences . . . . . . . . . . . . . . . . . . . . . . . . . .

54

2.4.3

Adaptation with respect to ellipsoids . . . . . . . . . . .

55

2.4.4

Adaptation with respect to arbitrary `p -bodies . . . . .

56

Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

61

2.5.1 2.5.2

Functional analysis: from function spaces to sequence spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . .

61

Gaussian processes

63

. . . . . . . . . . . . . . . . . . . .

3 NON LINEAR GAUSSIAN MODEL SELECTION Pascal Massart

71

3.1

A general Theorem . . . . . . . . . . . . . . . . . . . . . . . . .

71

3.2

Selecting ellipsoids and `2 regularization . . . . . . . . . . . . .

76

3.2.1

Adaptation over Besov ellipsoids . . . . . . . . . . . . .

77

3.2.2

A first penalization strategy . . . . . . . . . . . . . . . .

79

3.2.3 3.3

3.4

`2 regularization . . . . . . . . . . . . . . . . . . . . . .

81

`1 regularization . . . . . . . . . . . . . . . . . . . . . . . . . .

84

3.3.1

Variable selection . . . . . . . . . . . . . . . . . . . . . .

85

3.3.2

Selecting `1 balls and the Lasso . . . . . . . . . . . . . .

86

Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

87

3.4.1

Concentration inequalities . . . . . . . . . . . . . . . . .

87

3.4.2

Information inequalities . . . . . . . . . . . . . . . . . .

96

3.4.3

Birgé’s Lemma . . . . . . . . . . . . . . . . . . . . . . .

98

4 BAYESIAN MODEL CHOICE Jean-Michel Marin and Christian Robert 4.1

4.2

101

The Bayesian paradigm . . . . . . . . . . . . . . . . . . . . . .

101

4.1.1

The posterior distribution . . . . . . . . . . . . . . . . .

101

4.1.2

Bayesian estimates . . . . . . . . . . . . . . . . . . . . .

104

4.1.3

Conjugate prior distributions . . . . . . . . . . . . . . .

104

4.1.4

Noninformative priors . . . . . . . . . . . . . . . . . . .

105

4.1.5

Bayesian credible sets . . . . . . . . . . . . . . . . . . .

106

Bayesian discrimination between models . . . . . . . . . . . . .

107

Model Choice and Model Aggregation, F. Bertrand - Editions Techip  

For over fourty years, choosing a statistical model thanks to data consisted in optimizing a criterion based on penalized likelihood (H. Aka...

Model Choice and Model Aggregation, F. Bertrand - Editions Techip  

For over fourty years, choosing a statistical model thanks to data consisted in optimizing a criterion based on penalized likelihood (H. Aka...

Advertisement