Some of my recent work has used “interior” data to compute metrics which are then analysed by regular time-series methods. I've recently been thinking about paths to generalize this methodology for the analysis of high-frequency data.

The above diagram, I hope, illustrates what I mean by interior data. With regular time-series analysis we typically look at price changes over a homogeneous sequence of intervals and construct linear functions of lagged price changes. This is represented by the lower half of the data.

With higher frequency data, it is easy (i.e. cheap) to obtain data that summarizes trading activity down to resolutions of around one minute via the standard structure of “price bars.” The problem with this data is that, as the data frequency is increased, the data sequence becomes sparse, i.e. there are many intervals that do not contain trading activity, and quantized, i.e. we start to see the fundamental pricing interval of $0.01 strongly. Both of these factors disrupt the utility of classic time-series analysis.

These factors push one to analyze larger time intervals in order that a decent quantity of acceptably continuous data exists. With that constraint, it is easy to then only look in the direction of the data sampled at these larger time intervals. However, there is a set of data that exists within the interior of the larger time intervals, and my thoughts are that we should be able to assemble some kind of non-linear analytical machine to process the set of interior state vectors and forecast the evolution of the exterior process from that.

The correlation of the returns of the market (specifically the S&P 500 Index) and the VIX is large and negative, sampling in the region −85% to −100%. Without the results of the prior post, we are forced to seek a behavioural root for this observation:

demand for hedging instruments increases when the market is falling, increasing implied volatilities as the price of put options increases beyond fair value.

i.e. We have ascribed the observed covariance as a consequence of market participants' irrationality, as they believe that it's worth hedging only after they've lost money. However, Completely Asymmetric GARCH, such as that exhibited in the prior post, describes a process in which increases in future actual volatilities are only due to downwards moves and volatilities decrease after upwards moves. This linkage describes negative correlation between the returns of the index and future volatility and also describes a process which cannot crash upwards, as many commentators observe that the market does not do.

Why is this important? Well, consider if one believes that the increase in options prices on a down day is due to market participants situational bias. i.e. Their irrational belief that because the market is falling today then it is more likely to fall tomorrow. Under this scenario, the observed phenomenology represents an alpha or a predictable conditional mean for the future distribution of option trading profits. However, if the observations are consistent with the actual empirical price process — that down days do tend to increase future volatility — then there is no alpha. The increased prices represent an increase in fair value.

Under the first scenario, the right thing to do is to sell options; whereas, under the second, the right thing to do is to buy options. 

In prior posts we have demonstrated relative skill of 18% in making out-of-sample forecasts of the Dynamic Trading Risk Factor, a factor series that we hypothesize is a driving factor for the returns of companies that make money by dynamically trading securities and earning a premium from by selling the synthetic options created by such trading activity to their investors. We have also shown that the returns of well known public  financial companies, such as Goldman Sachs and Berkshire Hathaway are well explained by this factor. By well explained we mean that a linear regression of the monthly returns to investors, including dividends if any, onto the factor has a significant β and a large R². Exhibited below is our up-to-date chart comparing the monthly returns of Goldman Sachs, on which the rest of this analysis will concentrate, with those of the Dynamic Trading Risk Factor.

We originally performed this analysis in February, 2009; so in the following I will treat the period 2001:01–2009:01 as in-sample and 2009:02–2010:03 as out-of-sample. Using the Jackknife procedure discussed earlier, we find a bias corrected skill of 4% ±14% relative to the forecasts computed from the in-sample average monthly returns. The Dynamic Trading Risk Factor based forecasts are obtained by using the α and β, as established by linear regression within the in-sample period, as the model coefficients and the out-of-sample conditional mean factor forecasts from the AR(1) model for the driving factor series.

Using the metrics established in Grinold & Kahn's Active Portfolio Management, this forecast has an Information Coefficient, or IC of 23% which would lead to a Sharpe Ratio of 0.8 if traded on a monthly basis (I used G&K's  “rule of thumb” SR≈IC√N to estimate the Sharpe Ratio).

Standing alone, this skill estimate is not statistically signficant, but we know we have skill in forecasting the factor series and we know that the out-of-sample α and β for Goldman Sachs are consistent with their in-sample estimates, so my best guess is that the skill is weak but real.

I don't have much fresh data analysis of late as I am currently working to put together a new trading system. This system trades every 7½ minutes, using data I that I capture directly from my broker for pricing. Changing from the once-per-day style of the Compact Model Portfolio and A Poor Man's Hedge Fund is requiring quite a lot of work, particularly on the operational side.

I'm a great believer in getting into the market and letting the reality of the world sort out which of your ideas are right and which are wrong. For me, a critical parameter is the probability of executing a trade in which one attempts to buy at the current bid or sell at the current ask.

In this day and age, particularly in a world full of predatory high frequency trading algortithms, whos business is to run you up or down and fool you into paying a penny more than you would otherwise — and a strategy that makes pegging an order to the best bid or best ask particularly foolish — one needs to be very disciplined and cautious when designing a limit order trading strategy. The fastest players can cancel their bid and send an offer to sell to you within microseconds, leaving you having been stepped up the overpayment ladder one cent at a time.

Thus I do some experimental trading, by hand, as a man with a mouse and Excel™, to see what the market's really like. At a 7½ minute cadence, this leads to a tiring day of clicking.

And while I click, I try to think about the pattern's I'm observing. With this frequency of operation, I have about 50 opportunities per day to learn something — and fifty is equivalent to 2½ months on a daily basis — so there's sufficient statistics to learn a little in one day.

Yesterday, I found it difficult to get fills in the first fifteen minutes of the day. I immediately fell into the magical thinking trap of supposing that these fifteen minutes were somehow special; and more, in fact that the whole system was spuriously contrived around that period and was there becuase one cannot trade it during that time; etc. — as if a regression result that is driven by just 4% of the data is also weak enough not to be seen by the naked eye.

I find that quantitative traders can be quite prone to this kind of magical thinking. Despite building our systems, supposedly based on science, we all know that the NASDAQ is not the LHC and that our models are not reality but representations of reality and we fear that they may be just luckly flukes of the data. Thus we think that if we don't follow our prescriptions literally byte by byte, we will break a system that, in our hearts, we feel has no business being real in the first place. Even though our analysis suggests general descriptions such as industry groups tend to trend intraday, we end up feeling that really they trend from 9:34:34 am to 15:34:23 pm and any trespass into that first four minutes of the day is the moral equivalent of breaking a Faustian Pact.

I've worked as a sole proprietor and in small and larger companies over the last fifteen years. In these environments I've experienced different managerial structures as related to the practice of producing quantitative trading systems. My experiences have lead me to believe that Rapid Strategy Development is the best operational model for a quantitative research to follow.

Rapid Strategy Development means chosing to create a platform that removes roadblocks to the streamlined implementation of trading systems. The typical system development path can be defined as follows:

• empirical study of the market to generate ideas;
• use of analytical methods to specify viable ideas on historical data;
• use of analytical methods of verify viable ideas on newly captured data;
• definition of trading systems to implement the strategy;
• implementation of the codebase to execute the strategy.

The first two parts, the empirical and statistical research to create a system, should be the overwhelmingly longest part of the development path. The final stage should be the shortest. Once it has been researched, one should be able to code and execute a strategy within a day, that is the primary goal of the RSD paradigm. This really boils down to three principles.

1. Think First;
2. Avoid Ownership;
3. Just Do It.

Think First means the early recognition and implementation of generalizable structures (think before you type). Don't oppose generalization because it is “inefficient” — it is not.

Avoid Ownership means that my code is everybody's code and that everybody's code is my code. This means

1. accept input from others;
2. create of public use code (i.e. provided to other systems) over private use code (i.e. local to a specific system) whenever possible;
3. create of public use data tables whenever possible;
4. avoid of ad hoc nomenclature always;
5. avoid of non-conforming private data tables and private data storage whenever possible.

Just Do It (sometimes referred to in economics as "The Nike Strategy") means:

1. don't wait for permission to continue;
2. don't make private codes and seek to generalize them later "when I have the time" as that time will not come;
3. always compartmentalize to allow component reuse;
4. always make changes when their need is recognized — Fix It Now.

Professional managers are fully awhere of the transient and random nature of the returns they create, whether actively or passively, and are real human beings with the behavioural biases and oddities that characterize us as a group. Thus, when we are presented with a month in which we do very well, we are aware that the future will likely hold periods of underperformance. Furthermore, it is likely that the month following a good month, the month during which we are preparing a formal summary of the prior returns that we know were good, we are more likely to underperform that recent history than outperform it. Nobody wants to write the letter:

Dear Investor, last month we did very well. However, as I write this I know that we're doing less well, so don't get too carried away with your newfound wealth that I've already lost.

Furthermore, a manager who is confessing to a particularly dire prior period of returns would greatly like to write:

Dear Investor, last month we did badly. However, as I write this I know that we're doing very well, so please do not distress too much over your losses, which have already been erased.

For an example of this latter tendency, I can simply refer to my prior post on the September, 2009, performance of our NASDAQ-100 futures trading system. Both these forces together, provide the incentive for outperforming managers to report their returns promply and for underperforming managers to linger a while before sending the letters out of the door. Thus, we can explain the tendency observed in our analysis of the incremental updates of the BarclayHedge data.