Friday, 12 July 2013

The systemic bias of run-style in racehorse analysis: surviving the Late-Headway Monkey

An understanding of sectional times comes a long way down the list of important capabilities of the successful punter. At the top, in my opinion, is money management and staking, for even a bettor who has a positive expectation will go busto if failing to understand the potentially devastating effect of variance.

But sectionals do provide the opportunity to underline some long-held beliefs about racing and to revise others. And the knock-on benefit is that by delving deeper into the sport you will fire your interest and imagination even more, providing vital mental energy with which to unlock races under study, or to see through the kind of received wisdom against which others have no defence.

It only requires a basic understanding of sectionals to expose the biggest systemic flaw of analysis in British and Irish racing today: the late-headway bias. Think of it like this:

Before a race starts, a jockey's job is to maximise his mount's chance of winning - not to "achieve the best possible placing" which should be a lesser (though not unrelated) concern: it is clearly better to finish last while trying to win than to finish second while having no possible chance of victory. The problem is that the jockey (I am going to use the masculine from now on only for convenience) doesn't know exactly how the race is going to be run - even after discussion with his colleagues, for he could easily be the subject of deception.

Will it be an advantage to lie close to the pace? Or will it be deleterious to his chance? It is difficult in most races to adopt both extremes of position without disrupting a horse's psychological flow. What is clear, however, is that at some stage a jockey has to gain lengths on his rivals to win.

Imagine a series of races between cloned horses possessing equal ability, speed and stamina and no particular need to make the running or be given a lead. If this series continued to infinity, the sum of all results would correlate perfectly with the skill of the jockeys. If the races were on a straight course, the jockey who used his mounts' energy most efficiently would come out on top; if they were on a round course, the extra dimension to consider for the jockeys would be to save ground.

So how, exactly, does a jockey achieve the best possible win-rate? The ability of a racehorse can be thought of as a reserve of funds with which a jockey must purchase lengths. The rider on the best horse has the most funds to achieve victory, just as Manchester United, Chelsea and Manchester City have the most funds to win the Premiership. But it still has to be done.

When the speed of the race is relatively low, lengths are relatively cheap and a jockey should buy them and try to gain ground. But when the speed of the race is high, lengths become expensive and a jockey should adopt a more cautious approach, selling lengths at the top of the market.

The biggest change in the cost of lengths is at the start of the race. In the first couple of furlongs, the rate of change of speed is at its highest, from zero - as the horses are standing in the stalls - to somewhere near the optimal speed for that particular course and distance, often referred to as "cruising speed".

During this transition, a jockey intent on the lead must purchase lengths like a trader panic-buying on the floor of the old London Stock Exchange. All the time, the price is rising, making it difficult to know how expensive lengths are going to end up. All a superior rider can do is recognise the price-point at which buying lengths has to be a bad strategy and allow one of the maniacs of his profession to go on.

Riders adopting hold-up tactics have an equally shaky grasp of how much lengths will cost the leaders early on. However, by dropping out they are committed to a short-selling strategy which can turn out equally badly if the leaders get away with a slow pace up front. The hope of the hold-up rider is that the cost of lengths will either be so expensive early that it is bound to tumble eventually as the leaders tire, or that the pace will pick up too far from home. In either case, a hold-up rider will be able to purchase lengths in a fire-sale, gaining ground rapidly towards the end of a race.

So, which strategy tends to be the more effective? Buying lengths early like Paul Hanagan prefers, or buying them late after the manner of Jamie Spencer? It is easy to discern what most people think, judged by the disproportionate criticism which Spencer receives, but let's look at the problem in terms of its game-theory.

Game theory is the study of strategic options and their payoffs in a conflict situation, in this case the competition to win a race. It is concerned with the calculation of the expected value of every possible strategy and its counter-strategy, the cause and effect of winning and losing.

In general, jockeys are incentivised to win not just by prize-money and associated kudos but by the fear of being blamed for defeat: it is perceived as a serious error to finish with funds remaining and a trade-defecit of lengths. In this case, a hold-up jockey risks approbation by the horse's trainer, owner and those betting and writing about the race, all of whom have some influence over his reputation. Indeed, a mistimed ride can even result in a riding ban, if the perception that a rider's intentions were larcenous arouses the stewards.

So, jockeys as a group err on the side of overspending on lengths well before the finish is reached. And a point is often reached well before the line at which a collective desire to buy lengths leads to the price steepling and the formation of a "bubble". This is the point where race-readers often award the dreaded comment "every chance", indicating a horse was within striking-distance of the leaders - no matter how expensive the lengths cost to get there.

When a jockey is accustomed to a supply of superior mounts, he may reasonably adopt a sound default strategy to purchase lengths at the beginning of a race. Hanagan-style prominent tactics thus constitute what is called in game theory a minimax solution - the strategy which maximises the minimum possible gain and hedges against the greatest possible loss. For, if you have more resources to spend than your rivals, it is important to make sure the excess is put to good use. This is how Hanagan won two jockeys' titles, on a supply of superior mounts for trainer Richard Fahey.

In the majority of races, there may exist only one or two superior horses whose jockeys adopt this minimax solution of Hanagan, buying lengths early whatever their cost. But the more competitive a race becomes - defined here as a tightening in the spread of the ability of the horses - the less this works as a default strategy. The tendency is for the majority of riders to try to use all their resources well before the finish, so lengths are always expensive to gain some way from the finish and there is always a chance of a bubble, as all sense goes out of the window and jockeys buy lengths at prohibitive cost.

Importantly, bubbles are much less likely to affect a horse held up, unless the pace is very slow. When a jockey tried to gain ground late, he will often be doing it after the speed of the race has peaked. In other words, he will have adopted a "sell high-buy low" strategy with lengths and cannot go broke like the riders of front runners. Of course, if the cost of lengths was cheap when he sold them early, these tactics can backfire.

The overarching demand on jockeys not to leave it too late to buy lengths is exactly why the final furlong of a race is usually one of the slowest. Many of the jockeys have spent up and cannot buy lengths anymore, losing ground rapidly to hold-up jockeys who short-sold lengths in the first part of the race and still have resources left.

Now, let's consider the game-theoretical considerations of a jockey who adopts the default strategy of holding up. He can be pretty sure that lengths are going to be cheap right at the end of a race. What he risks is running out of time to purchase enough, and the resulting problems that giving his mounts too much to do causes for his career. So he, too, is incentivised not to leave it too late. The irritation for him, however, comes when his mount is simply not good enough to win: even when he sells and buys lengths for a sizeable profit, it may appear from his flying finish that he had excess funds remaining. And, more annoying still, his critics only see that he was buying lengths frantically late on. They make no allowance for the benefit which accrued by selling them cheaply in the first place.

By contrast, a jockey racing prominently who rides a bad race is often commended when he actually deserves to be blamed for defeat. If he makes a big move by buying lengths at peak cost, forces his mount to the front but is caught close home, the pundits say he has ridden an "enterprising race" or the even more egregious "got first run". The fact he made a sizeable loss by purchasing lengths at the top of the market and selling them cheaply is hardly ever mentioned.

The benefit to strong-finishing riders of selling lengths is often discounted by the critics because of the recency bias: replays of races focus on the last two furlongs, inviting us to think that what went on before was an abstract preamble; while the latest information relayed by our senses always causes the greatest impact, particularly when money is on the line. It is extremely difficult for analysts and punters to react positively to jockeys that deliver horses late and get beaten because most observers either don't want to understand the situation or simply don't have the requisite depth of knowledge. And it is always much easier to play to the crowd.

Sadly, there will always be larcenous rides in this great game of ours. And there will be many, many bad hold-up rides too, when jockeys sold lengths cheaply in the early stages and tried to buy them back expensively in the hottest part of the race, or when running wide. But jockeys who fail to win when gaining late tend to receive far too much blame - and their mounts far too much praise. It is stated liberally that they "stayed on from a poor position" or "had too much to do" because the analyst knows how easily such an excuse is accepted. Fewer riders are castigated for going too soon, however.

Now, a final word to those of statistical persuasion who might think that the higher Impact Value (IV) of front runners (their strike-rate considering normal expectations) denies the truth of the above. Rest assured: it doesn't.

Horses who are in the lead early win more often than their odds imply not because the tactics applied are the cause of victory but because they are the effect of a horse having superior ability. Think of a greyhound race. The leader after the first bend wins far more often than its odds imply not because it is a sound strategy to make the running but because the capability to lead is the product of superior ability. Horse races are, of course, much more complex than greyhound races because riders are involved, but the same principle applies. And, remember the incentive for riders of knowingly superior horses to buy lengths early whatever their cost? So, the group of horses from whom the IV of front-runners is drawn should win many more times than those off the pace early, because the latter sample contains many horses who cannot run fast enough at any stage of the race to win.

If the higher IV of front-runners compared with those trailing (many of which are not held up but just cannot run fast enough) simply reflected the differential payoffs of contrasting strategic options, then the consequence is that all horses could get better by being ridden more prominently. And it is easy by reductio ad absurdum to see the corollary of that: with more horses contesting the early pace, the price of buying lengths early would rise sharply and the payoff associated with front running would drop. So the percentage of horses front-running and those held up who could adopt different strategies remains in tension: in other words, there is no greater payoff to be gained by riding horses more prominently as a default strategy. Individual cases where this does result in a horse showing improved form, of course, remain.

Next time someone tells you that a horse caught the eye running on late, don't react with your gut; don't fall prey to the recency bias; don't run with the crowd. Instead, carry out a cost-benefit analysis of the tactics applied - when was the pace really at its strongest, according to the sectional times not the eye? And remember how cheap those lengths gained in the final furlong tend to be.

Think again: was it really a bad ride? Was it really unlucky? Or was it that evil simian who will eat up your funds like ripe bananas? Have you fallen prey to the Late-Headway Monkey?