Friday, 12 July 2013

The systemic bias of run-style in racehorse analysis: surviving the Late-Headway Monkey

An understanding of sectional times comes a long way down the list of important capabilities of the successful punter. At the top, in my opinion, is money management and staking, for even a bettor who has a positive expectation will go busto if failing to understand the potentially devastating effect of variance.

But sectionals do provide the opportunity to underline some long-held beliefs about racing and to revise others. And the knock-on benefit is that by delving deeper into the sport you will fire your interest and imagination even more, providing vital mental energy with which to unlock races under study, or to see through the kind of received wisdom against which others have no defence.

It only requires a basic understanding of sectionals to expose the biggest systemic flaw of analysis in British and Irish racing today: the late-headway bias. Think of it like this:

Before a race starts, a jockey's job is to maximise his mount's chance of winning - not to "achieve the best possible placing" which should be a lesser (though not unrelated) concern: it is clearly better to finish last while trying to win than to finish second while having no possible chance of victory. The problem is that the jockey (I am going to use the masculine from now on only for convenience) doesn't know exactly how the race is going to be run - even after discussion with his colleagues, for he could easily be the subject of deception.

Will it be an advantage to lie close to the pace? Or will it be deleterious to his chance? It is difficult in most races to adopt both extremes of position without disrupting a horse's psychological flow. What is clear, however, is that at some stage a jockey has to gain lengths on his rivals to win.

Imagine a series of races between cloned horses possessing equal ability, speed and stamina and no particular need to make the running or be given a lead. If this series continued to infinity, the sum of all results would correlate perfectly with the skill of the jockeys. If the races were on a straight course, the jockey who used his mounts' energy most efficiently would come out on top; if they were on a round course, the extra dimension to consider for the jockeys would be to save ground.

So how, exactly, does a jockey achieve the best possible win-rate? The ability of a racehorse can be thought of as a reserve of funds with which a jockey must purchase lengths. The rider on the best horse has the most funds to achieve victory, just as Manchester United, Chelsea and Manchester City have the most funds to win the Premiership. But it still has to be done.

When the speed of the race is relatively low, lengths are relatively cheap and a jockey should buy them and try to gain ground. But when the speed of the race is high, lengths become expensive and a jockey should adopt a more cautious approach, selling lengths at the top of the market.

The biggest change in the cost of lengths is at the start of the race. In the first couple of furlongs, the rate of change of speed is at its highest, from zero - as the horses are standing in the stalls - to somewhere near the optimal speed for that particular course and distance, often referred to as "cruising speed".

During this transition, a jockey intent on the lead must purchase lengths like a trader panic-buying on the floor of the old London Stock Exchange. All the time, the price is rising, making it difficult to know how expensive lengths are going to end up. All a superior rider can do is recognise the price-point at which buying lengths has to be a bad strategy and allow one of the maniacs of his profession to go on.

Riders adopting hold-up tactics have an equally shaky grasp of how much lengths will cost the leaders early on. However, by dropping out they are committed to a short-selling strategy which can turn out equally badly if the leaders get away with a slow pace up front. The hope of the hold-up rider is that the cost of lengths will either be so expensive early that it is bound to tumble eventually as the leaders tire, or that the pace will pick up too far from home. In either case, a hold-up rider will be able to purchase lengths in a fire-sale, gaining ground rapidly towards the end of a race.

So, which strategy tends to be the more effective? Buying lengths early like Paul Hanagan prefers, or buying them late after the manner of Jamie Spencer? It is easy to discern what most people think, judged by the disproportionate criticism which Spencer receives, but let's look at the problem in terms of its game-theory.

Game theory is the study of strategic options and their payoffs in a conflict situation, in this case the competition to win a race. It is concerned with the calculation of the expected value of every possible strategy and its counter-strategy, the cause and effect of winning and losing.

In general, jockeys are incentivised to win not just by prize-money and associated kudos but by the fear of being blamed for defeat: it is perceived as a serious error to finish with funds remaining and a trade-defecit of lengths. In this case, a hold-up jockey risks approbation by the horse's trainer, owner and those betting and writing about the race, all of whom have some influence over his reputation. Indeed, a mistimed ride can even result in a riding ban, if the perception that a rider's intentions were larcenous arouses the stewards.

So, jockeys as a group err on the side of overspending on lengths well before the finish is reached. And a point is often reached well before the line at which a collective desire to buy lengths leads to the price steepling and the formation of a "bubble". This is the point where race-readers often award the dreaded comment "every chance", indicating a horse was within striking-distance of the leaders - no matter how expensive the lengths cost to get there.

When a jockey is accustomed to a supply of superior mounts, he may reasonably adopt a sound default strategy to purchase lengths at the beginning of a race. Hanagan-style prominent tactics thus constitute what is called in game theory a minimax solution - the strategy which maximises the minimum possible gain and hedges against the greatest possible loss. For, if you have more resources to spend than your rivals, it is important to make sure the excess is put to good use. This is how Hanagan won two jockeys' titles, on a supply of superior mounts for trainer Richard Fahey.

In the majority of races, there may exist only one or two superior horses whose jockeys adopt this minimax solution of Hanagan, buying lengths early whatever their cost. But the more competitive a race becomes - defined here as a tightening in the spread of the ability of the horses - the less this works as a default strategy. The tendency is for the majority of riders to try to use all their resources well before the finish, so lengths are always expensive to gain some way from the finish and there is always a chance of a bubble, as all sense goes out of the window and jockeys buy lengths at prohibitive cost.

Importantly, bubbles are much less likely to affect a horse held up, unless the pace is very slow. When a jockey tried to gain ground late, he will often be doing it after the speed of the race has peaked. In other words, he will have adopted a "sell high-buy low" strategy with lengths and cannot go broke like the riders of front runners. Of course, if the cost of lengths was cheap when he sold them early, these tactics can backfire.

The overarching demand on jockeys not to leave it too late to buy lengths is exactly why the final furlong of a race is usually one of the slowest. Many of the jockeys have spent up and cannot buy lengths anymore, losing ground rapidly to hold-up jockeys who short-sold lengths in the first part of the race and still have resources left.

Now, let's consider the game-theoretical considerations of a jockey who adopts the default strategy of holding up. He can be pretty sure that lengths are going to be cheap right at the end of a race. What he risks is running out of time to purchase enough, and the resulting problems that giving his mounts too much to do causes for his career. So he, too, is incentivised not to leave it too late. The irritation for him, however, comes when his mount is simply not good enough to win: even when he sells and buys lengths for a sizeable profit, it may appear from his flying finish that he had excess funds remaining. And, more annoying still, his critics only see that he was buying lengths frantically late on. They make no allowance for the benefit which accrued by selling them cheaply in the first place.

By contrast, a jockey racing prominently who rides a bad race is often commended when he actually deserves to be blamed for defeat. If he makes a big move by buying lengths at peak cost, forces his mount to the front but is caught close home, the pundits say he has ridden an "enterprising race" or the even more egregious "got first run". The fact he made a sizeable loss by purchasing lengths at the top of the market and selling them cheaply is hardly ever mentioned.

The benefit to strong-finishing riders of selling lengths is often discounted by the critics because of the recency bias: replays of races focus on the last two furlongs, inviting us to think that what went on before was an abstract preamble; while the latest information relayed by our senses always causes the greatest impact, particularly when money is on the line. It is extremely difficult for analysts and punters to react positively to jockeys that deliver horses late and get beaten because most observers either don't want to understand the situation or simply don't have the requisite depth of knowledge. And it is always much easier to play to the crowd.

Sadly, there will always be larcenous rides in this great game of ours. And there will be many, many bad hold-up rides too, when jockeys sold lengths cheaply in the early stages and tried to buy them back expensively in the hottest part of the race, or when running wide. But jockeys who fail to win when gaining late tend to receive far too much blame - and their mounts far too much praise. It is stated liberally that they "stayed on from a poor position" or "had too much to do" because the analyst knows how easily such an excuse is accepted. Fewer riders are castigated for going too soon, however.

Now, a final word to those of statistical persuasion who might think that the higher Impact Value (IV) of front runners (their strike-rate considering normal expectations) denies the truth of the above. Rest assured: it doesn't.

Horses who are in the lead early win more often than their odds imply not because the tactics applied are the cause of victory but because they are the effect of a horse having superior ability. Think of a greyhound race. The leader after the first bend wins far more often than its odds imply not because it is a sound strategy to make the running but because the capability to lead is the product of superior ability. Horse races are, of course, much more complex than greyhound races because riders are involved, but the same principle applies. And, remember the incentive for riders of knowingly superior horses to buy lengths early whatever their cost? So, the group of horses from whom the IV of front-runners is drawn should win many more times than those off the pace early, because the latter sample contains many horses who cannot run fast enough at any stage of the race to win.

If the higher IV of front-runners compared with those trailing (many of which are not held up but just cannot run fast enough) simply reflected the differential payoffs of contrasting strategic options, then the consequence is that all horses could get better by being ridden more prominently. And it is easy by reductio ad absurdum to see the corollary of that: with more horses contesting the early pace, the price of buying lengths early would rise sharply and the payoff associated with front running would drop. So the percentage of horses front-running and those held up who could adopt different strategies remains in tension: in other words, there is no greater payoff to be gained by riding horses more prominently as a default strategy. Individual cases where this does result in a horse showing improved form, of course, remain.

Next time someone tells you that a horse caught the eye running on late, don't react with your gut; don't fall prey to the recency bias; don't run with the crowd. Instead, carry out a cost-benefit analysis of the tactics applied - when was the pace really at its strongest, according to the sectional times not the eye? And remember how cheap those lengths gained in the final furlong tend to be.

Think again: was it really a bad ride? Was it really unlucky? Or was it that evil simian who will eat up your funds like ripe bananas? Have you fallen prey to the Late-Headway Monkey?

Wednesday, 27 March 2013

Nonlinear dynamics of racehorse ratings: how upside can dominate exposed ability

Recall that 50,000 simulations of a race between five similarly exposed older horses produced the following results:
 
Horse
Rating
W
L
%
Anchovy
90
16307
33698
32.6
Butler
89
12536
37469
25.1
Coconut
88
9553
40452
19.1
Donkey
87
6740
43265
13.5
Einstein
86
4869
45136
9.7

The relationship between a horse's rating and its winning percentage in this framework can be seen as nonlinear. Start at the bottom with Einstein: if his trainer could improve him by one point, his chance of winning would go from 9.7% to 13.5%, the same as Donkey. We say there is a marginal improvement of 3.8% at a marginal cost of 1 point.

The next marginal change of 1 point takes Einstein from Donkey to Coconut, as it were. Reading from the table this is 19.1% minus 13.5% or 5.6% - a bigger marginal improvement than from the step from Einstein to Donkey.

And so, in this example, there is an exponential growth in the marginal benefit to a horse's chance for every marginal point that it improves. But there is obviously an upper limit, a bound, to this imposed by the fact that no horse can have a greater than 100% chance of winning.

So, let's run the simulation again, holding all things equal expect that the difference between horses is now 2 points rather than 1 point. Coconut is still the same 88 horse but his rivals are spread out more in terms of ability; there is more variance in their exposed merit, in other words:

Horse
Rating
W
L
%
Anchovy
92
19317
30688
38.6
Butler
90
13265
36740
26.5
Coconut
88
8642
41363
17.3
Donkey
86
5439
44566
10.9
Einstein
84
3342
46663
6.7

Compare the two tables, there is a significant difference in winning percentages at the extremes: Anchovy has seen his chance improve from 32.6% to 38.6% while Einstein has seen his deteriorate from 9.7% to 6.7%.
Compared with Coconut - the horse who owns an average rating in this simulation - Anchovy had a 13.5% better chance in the first example (32.6% minus 19.1%) but has a 21.3% better chance (38.6% minus 17.3%) this time. But, as neoclassical economics dictates, let's look at the changes in percentage chance of success at the margin.
In the first case, the 13.5% better chance came at a marginal cost of two points (from Coconut's rating of 88 to Anchovy's rating of 90). So, the marginal benefit of one point is 6.25%.
In the second example, however, the marginal improvement of 21.3% (Anchovy's 38.6% minus Coconut's 17.3%) came at a marginal cost of four points. So, the marginal benefit of one point is only 5.325%. 
I want to flesh out this important point more. So, let's now increase the difference between horses to 5 points, so now poor old Einstein is 20lb inferior to Anchovy. (Perhaps the latter just joined Satish Seemar in Dubai.)
Horse
Rating
W
L
%
Anchovy
98
27652
22353
55.3
Butler
93
13533
36472
27.1
Coconut
88
5876
44129
11.8
Donkey
83
2232
47773
4.5
Einstein
78
712
49293
1.4

Now, Anchovy is 5-4 on favourite and Einstein is just about a 100-1 poke. This example perhaps resembles a Group race whereas our first simulation could serve as a proxy for a handicap. Let's re-examine the marginal benefit of improving from Coconut (rating 88 again) to Anchovy (now rated 98).

The marginal improvement is 43.5% at a marginal cost of 10 points. So, the marginal benefit of one point is now only 4.35%. It is obvious that the more spread out the ratings become, the less benefit there is to a horse's chance from a marginal improvement of one point. Hopefully, this is exactly what most punters would intuitively understand.

However, it is important to remember we are dealing with five horses who have exactly the same potential for improvement. If you remember the last blog, I noted that, in some situations where the sub-populations of horses are not connected - such as three-year-olds and older horses on the Flat, Dubaian form and European form, or novice hurdlers in Ireland and British handicap hurdlers -  a horse's exposed ability is negatively correlated with its potential for improvement. (This was rephrased in several of the superb Twitter responses I received subsequently in the more familiar terms of horses being exposed/unexposed.)

So, now let's give one of our horses, Butler, greater upside than its rivals. To do this mathematically from my original sample of the Beyer speed figures of older horses in the US, I simply divided the population into sub-populations of horses who had a difference in average starts of five and recalculated the parameters of the distribution from which the Monte Carlo simulation makes a random draw. (For the technically minded, this second population had a fatter right-tail and a lower peak, representing a greater chance of ratings distant from the mean and a higher standard deviation).

Now, Butler belongs to this lighter-raced group and Anchovy, Coconut, Donkey and Einstein remain as they were. Let's go back to the first example, the handicap-style encounter between closely matched rivals in which their median ratings are separated by only a point. Here is another reminder of that initial out turn of 50,000 simulations:

Horse
Rating
W
L
%
Anchovy
90
16307
33698
32.6
Butler
89
12536
37469
25.1
Coconut
88
9553
40452
19.1
Donkey
87
6740
43265
13.5
Einstein
86
4869
45136
9.7

Now, look what happens when Butler belongs to a cohort of horses who have raced five less times on average:

Horse
Rating
W
L
%
Anchovy
90
14832
35173
29.7
Butler
89
15797
34208
31.6
Coconut
88
8628
41377
17.3
Donkey
87
6255
43750
12.5
Einstein
86
4493
45512
9.0

 ...with the following placings:

1
2
3
4
5
14832
11507
8438
7967
7261
15797
9278
8187
8482
8261
8628
10990
10646
9925
9816
6255
9839
11306
11248
11357
4493
8391
11428
12383
13310

 ...producing the following parameters:
AVG
STD
MAX
MIN
MEDIAN
87.4
7.90
108
48
90
87.3
8.45
115
40
89
85.4
7.98
106
37
88
84.4
7.83
107
39
87
83.4
7.89
104
40
86
Butler is now the favourite! I hope you find this as thought-provoking as I did the first time I ran these numbers. Leaving aside the particular dynamics of the older horse population in the US on dirt, the most important takeaway point is as follows:

Anchovy has a higher rating than Butler going into the race (90 to 89) and the difference in their median figures is this same 1 point in 50,000 races. Anchovy's average rating (87.4) is also still higher than Butler's (87.3) but because the latter has more upside (represented by a higher standard deviation of his performance level (8.45 to 7.90) and a higher peak effort (115 to 108) in rare situations, he is actually more likely to win a race between them (making all the same assumptions I laid out in the last blog).

Remember, the change to Butler's distribution of expected ratings was not dramatic in and of itself. Here are some situations in British racing which would represent much bigger likely swings between horses that have had different numbers of starts or opportunities to achieve their best ratings:

  • Two horses in a maiden when one has run three times and the other once
  • Nursery handicaps in which the top weight has beaten horses with low ratings while its opponents have been beaten by rivals with bigger ones
  • Irish novice hurdlers against British ones that have run in the Betfair Hurdle against exposed handicappers
  • Dubai-trained horses who have thrashed locals and are now facing fancy outsiders with big figures in Group 1 races round the world

 I'm certain you can list many other situations in which the opportunity cost of achieving a rating has not been discounted before projecting a horse's chance.

If you were confused by what I was saying about My Tent Or Yours, no implication was being made about his Betfair Hurdle form or the strength of the Supreme Novices' form or whether you would go skint laying horses who were clear top-rated at the Festival (whether you go skint has far less to do with your ability than your temperament, in any case.)

Instead, the point is that if you were trying to create a computerised SP and one of your inputs was (hopefully) a horse's form, there must be a term in the equation representing opportunity cost, so that the likely distribution of all its future ratings reflects the importance of prior opportunity. It is not an overcomplication, it is an extremely important correction to an oversimplification of some who make bogus inferences from figures, especially on the television.

What we are working towards here is a mathematical understanding of the importance of upside compared with exposed ability. Many of you whom I have met over the years have an extremely good intuitive grasp of this nonlinear dynamic, one ability which makes them a superb punter. My brain does not work like yours, unfortunately, in that I have to quantify my edge before having a bet.

I do not trust my instincts, partly because the single most dominant influence over my cognitive development was my maternal Grandfather who impressed on me the beauty of numbers and that everything in life - even those things assumed to be the preserve of the aesthete - were underpinned by quantitative processes. In some ways, he has made my life hell but mostly I feel lucky to have been inspired in this way. And, my mantra is that most things which come from your superior intuition are amenable to a test for cognitive bias. Or, as in the case of my friend Tom Segal, just evidence of a brilliant, intuitive mind.