When I started my career I was asked to analyze the performance of a volatility arbitrage system. One of the tools we looked at was the Market Information Machine (XMIM) produced by Logical Information Machines. In 1994 and 1995, which I when I was using the product, this was a data mining product for traders that allows one to simulate trading by rules such as

when gold is up more than 5% and oil is down more than 2% then sell gold

for example. The MIM people had put a lot of effort into crafting natural language queries, because back in the day traders were thought of as a group that could barely put together an Excel worksheet.

One of the outputs of the system was a statistical breakdown of winning trades and losing trades, in particular the number of winning trades versus the number of losing trades and this reminded me of all the work I had done with event rate counting when I was working in statistical astronomy with cosmic rays for my doctoral research.

Now, for a trading system the number of winning trades versus the number of losing trades is actually an irrelevant metric. What is important is the total dollars won versus the total dollars lost. I have worked with many trading systems over the past 15 years where there were always more losing trades than winning trades, but the winning trades paid out much more than the losing trades lost (this is often true of momentum strategies, for example).

However, for a humans the truth is that it is emotionally easier to deal with a system which makes money more often than it loses, rather than one with skewed payoffs that loses money most of the time and occasionally makes a lot of money. In fact, I would go so far as to suggest that one reason certain dynamic trading anomalies exist in the market, even though they are fairly easy to identify, is because they are so difficult to live with from the perspective of a human risk manager.

The prior philosophical discussion not withstanding, let's look at a statistic that allows us to assess whether there is an excess of up or down items (be it days, trades, stocks). Let's start off by reviewing the statistics of counting. Basically, if we are counting the occurrences of a random event, and that event is one which occurs at a characteristic rate, then the number of events that occur within a particular sample are drawn from the Poisson Distribution. This distribution is fairly basic, and can be derived as the consequences of a binary process (that the event either does or does not occur within an interval that is very small).

The key thing to remember about the Poisson distribution is that

if an event has an expected rate N per interval then the population standard deviation for the interval is √N

What this means from a statistical point of view is that

you need four times the data a achieve twice the accuracy in sampled measure

At the Soudan 2 Experiment, which was a proton-decay experiment, my collaborators were also interested in working on Atmospheric Neutrino Oscillation phenomena, which is the observed flavour change of the muon neutrinos created in extensive air showers, which are the result of very high energy cosmic ray impacts with the upper atmosphere.

We were looking to count the numbers of events that could be associated with electron neutrinos and muon neutrinos, and to compare those numbers to theory. Several people cast around for statistics to measure the (fairly low) event rates. We wanted a statistical that was standardized in the statistical sense. Most members used the fairly direct measure

I felt at the time, and still do, that this statistic treats both of its elements asymmetrically, and that this is unfair. Instead of something modelled on the the above, I like to work with an event statistic that looks like

This statistic does not favour one channel over the other and is, in the limit of large numbers, statistically standardized (i.e. the WL∼N(0,1) meaning that it is Normally Distributed with a zero mean and unit standard deviation.

Some might argue that in a binary state system — i.e. either you win or you lose — that this is not right as the N(wins) and N(losses) are the results of a binary choice that occurs a fixed number of times, and so we should use the Binomial Distribution to describe our samples. However, in a system where we do not trade on every signal, and which has the possibility of neither making nor losing money, this is not correct. For Barrier Trading Systems, the number of trades follows Poisson Statistics and the lack of a winning trade does not guarantee a losing trade — so my measure is ok. It is what I refer to as the Win-Loss Statistic in the charts and analyses on this site.

William Eckhardt is a very famous futures trader. He, along with Richard Dennis, set up the so-called Turtle Traders. Eckhardt runs the Eckhardt Trading Company, which is a large and long lived Managed Futures fund. The monthly returns for their Standard programme are available on the internet, and I have taken a copy of that data for analysis.

Our regression of the returns of this fund onto the dynamic trading risk factor give an α of 0.88 ± 0.34 %/month, with a p-Value of 1%; and, a β of 0.10 ± 0.17, which is 5.4σ from unity. These estimates seem to present us with the conclusion that Eckhardt is doing something different to the activities of typical hedge funds. Since most futures funds are trend followers, who do not do much in-and-out trading but work on the assumption that commodity prices follow gross macroeconomic trends, it is plausible that the risk factor they are exposed to is different to that of more typical hedge funds.

Clarium Capital Management is a large macro fund run by Peter Thiel. I looked at regressing the returns of the Clarium LP fund onto the dynamic trading risk factor in an earlier post.

Clarium seems to be one of the easier funds to get returns data for. In our earlier analysis we found that Clarium had a very large, but statistically marginal, α and a β statistically indistinct from unity, when regressed onto the dynamic trading risk factor. With an extra three months of data we shouldn't expect much to change. (The Clarium LP monthly returns data I am working with is available on the internet at sites such as Market Folly.)

Clarium is essentially flat on the year, whereas the typical hedge fund, as exemplified by the dynamic trading risk factor, is up on the year. Consequently, the α and the β have both declined slightly. The p-Value for the α is now 0.04.

Our predicted return for May, 2009, is a gain of 3.23%

This post really has nothing to do with statistical trading, but I'm including as a record of what I implemented since I could find very little help with this topic on the internet.

The basic issue is to redirect a blog so that new users, old users, and, search engines all get redirected to the new site. The official way is the so called 301 Redirect status returned by a web server. However, that method requires administrative control of the web server — not something that most users of free blog services actually have.

Asecond method is the meta refresh method, in which we insert a META tag, such as the one below, into the HEAD section of the home page. This is something that can be done with a free web service and will instruct a browser client to redirect to a new page. The one I used was:

<meta content="0;url=http://blog.gillerinvestments.com" http-equiv="refresh"/>.

This method will send browsers to the new site and also allow search engines to find the new site, as Google et al. will read the meta tag and follow the link to the new site.

However, it has the drawback in that it directs solely to the blog front page. If a user has followed a search engine's results to a post on the old site, they will loose their position within the blog, and maybe won't be inclined or able to search again to locate the document they wanted to read.

I decided to deal with this by adding a JavaScript function to be loaded by the BODY tag's onload event. This can be used to implement a redirect because web browsers respond to assignment to the window.location object's href item by loading the new URL.

<script>
function redirect_it()
{
  window.location.href=(document.title=="Statistical Trader"?
    "http://blog.gillerinvestments.com":
    "http://blog.gillerinvestments.com/search.aspx?q="+
      escape(document.title.slice(20)));
}
</script>

This script is triggered by the onload event as follows

<body onload="redirect_it();">

What the function does is a little application specific. In my implementation, it takes the page's title data and passes it to the search function on my new blog site. As I loaded all of my previous posts into the new blog, that should find the page that a redirected user was looking at. It would be better to mangle the permalinks, but my new system (BlogEngine.NET) does not use compatible slugs. As a final wrinkle, it just goes to the front page if the user was just looking at the front page.

Hopefully, this will not lose too many readers since all they need to do is make one more click to get to new location of the page that they had actually wanted to look at.

Following the Science 2.0 concept of making data available for others to repeat my analyses, I am sharing the history of my intraday index futures forecasts (the raw data used to support the analyses presented earlier). The forecast history for Mini-DJIA Index futures (YM) and the forecast history for Mini-NASDAQ-100 Index futures (NQ) are available.

This data is that used to create the charts described in the posts Forecasting Accuracy for the DJIA Intraday Strategy and Forecasting Accuracy for the NASDAQ-100 Intraday Strategy.

We have a new month and new data for the dynamic trading risk factor. Last month's forecast was for a profit of 1 %; the realization was a profit of 5.20 %. This month we are forecasting a profit of 2.17% for May, 2009. (Note that the underlying charts for the prior posts have been replaced by the newer versions.)

On the chart above I have, arbitarily, identified the start of the crisis as October, 2007; and, the end of the crisis as November, 2008. (The three regions are labelled pro articulus; per articulus; and, secundum articulus; repsectively.) This identification was done by "eyeballing" the peak and trough of the cumulative factor series without reference to any other criteria. Although this uses up two degrees of freedom, since we are choosing part of the data according to some criteria of our choice, I wasn't trying to choose regions according to the underlying properties of the data and I've not varied these choices to emphasis any statistic. I think, on this basis, I can legitimately use the two sample Kolmogorov-Smirnov test without violating the criteria regarding its validity. I'm not 100% confident of that, but I'm over 99% confident!

Without making any distributional assumptions, we can ask whether the pro articulus data set and the sec. articulus dataset are consistent with being drawn from the same underlying distribution. The two sample K-S test allows us to compare the empirical distribution function for two data samples and assess whether they are consistent with eachother without every having to specify the underlying population distribution. It is a truly remarkable tool.

The chart above presents the results of this analysis, including a chart comparing both empirical distribution functions. The Dmax statistic is 0.27901 with a sample of 81 months pro art. and 5 months sec. art. The test indicates that the distributions agree, with a p-Value of 0.91453 (indicating the probability of obtaining a larger Dmax by chance with samples this size). Again, the similarity of the two samples was not the criteria used to choose the samples — which is why I believe that this is an unbiased use of the test.

In plain terms, the data does not contradict the assumption that the returns are distributed as they were before.