For a while now I've made forecasts of the future returns of the Dynamic Trading Risk Factor and therefore, by proxy, the monthly returns of a typical hedge fund, based on a classic Box-Jenkin's style AR(1) model for the factor.

The purpose of this post is to analyse the relative skill exhibited by this forecast relative to two appropriate nulls. Those are:

1. The Law of Large Numbers Forecast — i.e. the mean of all the previous returns in the in-sample period, which is the data on which the AR(1) model was developed; and,
2. The Markov Process Forecast — i.e. the forecast based on the assumption that best estimate of the future returns is the return that just occurred.

For the purposes of comparing these forecasts we will use the commonly defined Forecasting Skill, being one minus the ratio of the mean square error of the proposed forecast to that of the null or “business as usual” forecast — which in our case will be the Law of Large Numbers forecast. This is based on the idea that, in the limit, the sample mean is an efficient and unbiased estimator of the population mean (for distributions for which the second moment exists).

Using these metrics we find that (entirely out-of-sample) the relative skill of a classic Box-Jenkins AR(1) model is 18% and the relative skill of the Markov Process model is −9%. A satisfying confirmation of the (well known) validity of the Box-Jenkins approach. For completeness, the skill of the AR(1) forecast relative to the Markov Process forecast is 25%.