Following on from May's difficult trading, we forecast a continued loss for the Dynamic Trading Risk Factor of 65 bp for June, 2010, and similar returns for correlated typical hedge funds. Early indications are that the numbers are coming in at a rate of −87 bp — marginally worse than the expectation. However, our early forecast for July, 2010, is now a slight gain of 7 bp.

In the prior post we presented evidence that the forecasting skill of our AR(1) model of the returns due to the dynamic trading risk factor was +18% relative to the null hypothesis forecast based on the sample mean return. However, we did not give a statement of the significance of this excess skill based on the sample data recorded.

The principal reason for this omission was that I don't know the sampling distribution of the skill statistic, and so cannot easily assess it's sample variance. In this post we will use the Jackknife, which is a statistical resampling technique, to estimate the bias and variance of the skill statistic.

For N data points, the basic technique is to compute the statistic we are interested in over the N subsets of the data that may be selected by leaving one of the data points out in each group. Unlike bootstrapping, we do not select subsets at random — we consider every possible subset that may be formed with just one datum left out. We may then use the sample distribution of these leave-one-out estimators of the skill to estimate the bias and variance of our whole sample statistic. 

Our data has a null forecast of 0.46% per month and a relative skill of 18%. From the data below, we compute a Jackknife bias of −2% in skill, leading to a Jackknife estimator of 20% skill with a Jackknife variance estimator of 0.0207 (std.err. of 14% in skill). 

For a while now I've made forecasts of the future returns of the Dynamic Trading Risk Factor and therefore, by proxy, the monthly returns of a typical hedge fund, based on a classic Box-Jenkin's style AR(1) model for the factor.

The purpose of this post is to analyse the relative skill exhibited by this forecast relative to two appropriate nulls. Those are:

1. The Law of Large Numbers Forecast — i.e. the mean of all the previous returns in the in-sample period, which is the data on which the AR(1) model was developed; and,
2. The Markov Process Forecast — i.e. the forecast based on the assumption that best estimate of the future returns is the return that just occurred.

For the purposes of comparing these forecasts we will use the commonly defined Forecasting Skill, being one minus the ratio of the mean square error of the proposed forecast to that of the null or “business as usual” forecast — which in our case will be the Law of Large Numbers forecast. This is based on the idea that, in the limit, the sample mean is an efficient and unbiased estimator of the population mean (for distributions for which the second moment exists).

Using these metrics we find that (entirely out-of-sample) the relative skill of a classic Box-Jenkins AR(1) model is 18% and the relative skill of the Markov Process model is −9%. A satisfying confirmation of the (well known) validity of the Box-Jenkins approach. For completeness, the skill of the AR(1) forecast relative to the Markov Process forecast is 25%.

In prior posts we have investigated whether it is possible to make an early forecast of the Dynamic Trading Risk Factor return for a specific month, and discussed our motivation for making an early forecast. In this post we examine a necessary condition for that work to be worthwhile — that the returns of an asset due to the factor are not delivered in a clump at the start of the month.

To make things easy for ourselves, we'll start with the Franklin Mutual Shares fund, which we have under the ticker TESIX. We found that this fund has a β of 1.7 onto the Dynamic Trading Risk Factor. With such a strong correlation, hopefully it will show a strong effect if one is there. (Of course, this is actually a bad choice for market timing because the front load fee of over 5% for this mutual fund will likely kill any timing alpha we can discover.)

Here we find a daily β of 2.3 with an apparent linear decay rate of (0.12 ± 0.06) per day. This result has borderline significance (p-Value of 3.5%) and so does not contradict the hypothesis that the factor returns are delivered uniformly throughout the month.

In the prior post we showed some skill in estimating the end-of-month factor returns for the Dynamic Trading Risk Factor based on early measurements of partial universe returns and a bias correction model.

However, we've not indicated why that would be interesting. The principal reason is as follows: we have good evidence that the Dynamic Trading Risk Factor trends — specifically that an AR(1) model can be fitted to it. This means that we can predict the next month's returns of the factor based on this months inferred value. But, and this is the problem, we typically only fully know a month's factor return at the end of the next month — and that's the month we're trying to predict.

If we could take an early estimate of the value we would record at the end of the current month, then we can make an estimte of the forecast for the current month with sufficient time to act upon that information. For example, we could time trades in one of the public listed securities that we have demonstrated have a high R² onto the factor.

Of course, for our market timing to work, it is necessary that the asset returns that are associated with the factor be delivered, to some extent, over the remaining part of the month. In the next post, we will seek to establish the validity of this proposition. 

In the previous post, we investigated whether outperforming funds report their results earlier in the month, and whether that induces a systematic bias into the early reported hedge fund indices relative to the finalized values at the end of the month. The table below shows my measurements of the Dynamic Trading Risk Factor, as estimated through the procedures outlined on this blog, as determined according to a fairly ad-hoc updating schedule determined by my commitments to other projects.

For each month we present the first recorded estimate of the factor return, the final recorded estimate of the factor return, and a “corrected estimate” or forecast final value based upon our bias expression from the prior post and the initial estimate. We compute residuals for two hypotheses:

1. the null, H₀, that the early measurement is an unbiased estimate of the final value; and,
2. the alternate, H₁, that the corrected estimate is a better forecast of this final value.

We can compare these hypothesis by evaluating their forecasting skill, which gives a 23% edge to the corrected estimate. This skill is usually defined by the following equation, where MSE means mean square error.