When I started my career I was asked to analyze the performance of a
volatility arbitrage system. One of the tools we looked at was the
Market Information Machine (XMIM) produced by Logical Information
Machines. In 1994 and 1995, which I when I was using the product, this
was a data mining product for traders that allows one to simulate trading by
rules such as
when gold is up more than 5% and oil is down more than 2%
then sell gold
for example. The MIM people had put a
lot of effort into crafting
natural language queries, because back in
the day traders were thought of as a group that could barely put together an
Excel worksheet.
One of the outputs of the system was a statistical breakdown of winning trades
and losing trades, in particular the number of winning trades versus the number
of losing trades and this reminded me of all the work I had done with event
rate counting when I was working in statistical astronomy with cosmic
rays for my doctoral research.
Now, for a trading system the number of winning trades versus the number of
losing trades is actually an irrelevant metric. What is important is the total
dollars won versus the total dollars lost. I have worked with many trading
systems over the past 15 years where there were always more losing trades than
winning trades, but the winning trades paid out much more than the losing
trades lost (this is often true of momentum strategies, for example).
However, for a humans the truth is that it is emotionally easier to deal with a
system which makes money more often than it loses, rather than one with skewed
payoffs that loses money most of the time and occasionally makes a lot of
money. In fact, I would go so far as to suggest that one reason certain dynamic
trading anomalies exist in the market, even though they are fairly easy to
identify, is because they are so difficult to live with from the
perspective of a human risk manager.
The prior philosophical discussion not withstanding, let's look at a statistic
that allows us to assess whether there is an excess of up or down items (be it
days, trades, stocks). Let's start off by reviewing the
statistics of counting. Basically, if we are counting the occurrences
of a random event, and that event is one which occurs at a characteristic rate,
then the number of events that occur within a particular sample are drawn from
the Poisson Distribution. This distribution is fairly basic, and can
be derived as the consequences of a binary process (that the event either does
or does not occur within an interval that is very small).
The key thing to remember about the Poisson distribution is that
if an
event has an expected rate N per interval then the population standard
deviation for the interval is √N
What this means from a statistical point of view is that
you need four
times the data a achieve twice the accuracy in sampled measure
At the
Soudan 2 Experiment, which was a proton-decay experiment, my
collaborators were also interested in working on
Atmospheric Neutrino Oscillation phenomena, which is the
observed flavour change of the muon neutrinos created in
extensive air showers, which are the result of very high
energy cosmic ray impacts with the upper atmosphere.
We were looking to count the numbers of events that could be associated with
electron neutrinos and muon neutrinos, and to compare those numbers to theory.
Several people cast around for statistics to measure the (fairly low) event
rates. We wanted a statistical that was standardized in the statistical
sense. Most members used the fairly direct measure
-N(e)}{\sqrt{N(\mu)}})
I felt at the time, and still do, that this statistic treats both of its
elements asymmetrically, and that this is unfair. Instead of something
modelled on the the above, I like to work with an event statistic that looks
like
-N(\textsl{losses})}{\sqrt{N(\textsl{wins})+N(\textsl{losses})}})
This statistic does not favour one channel over the other and is, in the limit
of large numbers, statistically standardized (i.e. the WL∼
N(0,1)
meaning that it is
Normally
Distributed with a zero mean and unit standard deviation.
Some might argue that in a binary state system — i.e. either you
win or you lose — that this is not right as the N(wins)
and N(losses)
are the results of a binary choice that occurs a fixed number of times, and so
we should use the
Binomial Distribution to describe our samples. However, in a
system where we do not trade on every signal, and which has the possibility of
neither making nor losing money, this is not correct. For Barrier Trading
Systems, the number of trades follows Poisson Statistics and the lack
of a winning trade does not guarantee a losing trade — so my measure is
ok. It is what I refer to as the Win-Loss Statistic
in the charts and analyses on this site.