Doctoral thesis

Essays on Expected Prediction Error

Finite-Sample Bias in Cross-Validation and Pseudo-Out-of-Sample Testing: This paper analyses finite-sample bias in cross-validation estimates of expected prediction error. A s ...

Author(s) :

Ian Hunt, PhD

Research Fellow at School of Mathematics of Monash University, Melbourne (Australia)

Abstract :

Finite-Sample Bias in Cross-Validation and Pseudo-Out-of-Sample Testing: This paper analyses finite-sample bias in cross-validation estimates of expected prediction error. A significant risk of positive bias against flexible models is identified— this bias has practical implications for assessing curve-fitting models in finance and economics, for example when comparing regime-change models with less flexible model-averaging methods. The bias against flexible models is also positively related to the number of data-points left out for cross-validation purposes — this has implications for pseudo-out-of-sample Diebold-Mariano tests because these tests rely on a special case of cross-validation that leaves out large chunks of data. I examine real models that predict equity market returns, in which all forms of cross-validation appear to be extremely biased against flexible models. And I present a simulation study which supports my analytic conclusions and is consistent with the equity return results.

Optimistic Equity Premium Predictions: I compare equity premium prediction models in terms of expected prediction error, using a new method that is less biased against flexible models than conventional out-of-sample analysis. I add to the literature by finding that: the best models can predict premiums in economic good-times, not just bad-times; successful premium predictions are driven by growth, rather than value, assets; the best models overall are also the best within all meaningful subsets of periods; and while models do well under a squared error loss function, they do not do well under absolute error or sign error loss functions.

Date : 16/03/2020
Thesis Committee :

Supervisor: Abraham Lioui, EDHEC Business School

External reviewer: Francis X. Diebold, University of Pennsylvania

Other committee member: Nikolaos Tessaromatis, EDHEC Business School

See Also

What were EDHEC-risk’s top 10 most read articles in 2021?
News
- 23-12-2021
A year in research: focus on retirement investing (goal-based investing,...
Paper by EDHEC-Risk Associate Researcher Daniel Mantilla forthcoming in Finance Research Letters
News
- 14-12-2021
WE ARE PLEASED TO ENCLOSE AN EDHEC-RISK INSTITUTE RESEARCH ASSOCIATE ARTICLE...
Professor Laurent Calvet awarded two prizes for advancing the body of knowledge for financial planning
News
- 13-12-2021
Laurent Calvet, Member of EDHEC-Risk Institute and Professor of Finance at EDHEC...
Chasing the ESG Factor by EDHEC Professor Abraham Lioui
News
- 08-12-2021
Is ESG Disagreement Material ? To answer this question, one needs first a methodology...