Introduction

On day two of the November 2013 Cheltenham meeting David Pipe drew a blank from his seven runners. For many people this result pointed to the Pipe stable being out of form. But whatever might have been ailing the yard on Saturday had disappeared by Sunday morning, with four winners, including The Greatwood, one of the most competitive handicap hurdles in the calendar. No doubt on Sunday evening the Pipe yard was marked out as one to follow. So what of trainer form, is it possible to identify yards that are in or out of form? In the woefully misnamed ‘Statistics’ section of the Racing Post the ‘Hot Trainers’ table uses Strike Rate (winners to runners) over the last 14 days , whilst  the ‘In Form’ table in the ‘Trainerspot GB’ table uses Run To Form, again over the last fortnight. Neither table is useful.  Both are based upon too few runners to be able to draw any meaningful conclusions. These tables are an excellent example of attempting to draw inference from a small information set – whilst this instinct helped us survive when faced with perceived mortal dangers in the past, the very same instincts are likely to mislead in the more prosaic setting of horse racing. Whilst the data in the  ‘In Form’ table isn’t useful, this is only because there are too few observations to be able to draw any firm conclusions. However, the idea of considering trainer form as an average or median of how close to form the horses under the trainers care are running makes intuitive sense.

The analysis that underpins this piece was carried out in the R statistical environment accessing Raceform Interactive data focussing on the 2010-11 and 2011-12 National Hunt seasons. Thanks to Simon Rowlands and James Willoughby for their input.

Trainer Form Variable Definitions

The starting point for the analysis that follows is to define and calculate a Run To Form (RTF) variable. RTF is defined as follows:  the Racing Post Rating (RPR) achieved by each horse in a race subtracted from the maximum RPR achieved by each horse in its runs up to and including the race in question. A horse has to have run more than three times to qualify for consideration. This filter is used to reduce the influence of lightly raced progressive horses. The maximum value RTF can take is zero.

Trainer Form Absolute (TFA)

The next step is to define and calculate a measure of trainer form. Trainer Form Absolute (TFA) is the median RTF for all the runners of a trainer over a particular time period.  Using the 2010-11 National Hunt season Graph 1 shows a histogram of TFA for those trainers that had at least 50 runners over the season. 132 trainers qualified. Note the negative skew. This is a common characteristic of form data in horse racing.  It is difficult to run close to form wheras there are many and varied reasons for horses running below form. The negative skew in RTF at the horse level aggregates to negative skew in TFA at the trainer level. Graph 2 shows the same information as a Box Plot.

Graph 1: Distribution of trainer TFA NH season 2010-11

Graph 2: Box Plot of trainer TFA NH season 2010-11

So does a ranking of TFA indicate in form trainers at the top and out of form trainers at the bottom? Inspection of Graphs 1 and 2 highlights the problem with using TFA as a measure of trainer form. The range of TFA across trainers is so wide that the top and bottom of the TFA list won’t change often enough to be able to identify trainers in and out of form – for example if Nicky Henderson normally runs at a -5lb TFA and is currently running at -8lb TFA he would still be near the top of the TFA list, wheras he is running 3lb below his normal TFA rate. It is also of note that the variability of TFA per trainer is correlated with their TFA. Either the better horses, who are at the better yards, run more consistently, or the better trainers are able to get their horses, who are better than those elsewhere, to run more consistently. Or a combination of the two. In this context better means those yards with the highest TFA values.

Trainer Form  Relative (TFR)

Given the concerns about using TFA a measure of relative run to form can be defined and calculated that takes into account the usual RTF per trainer. Trainer Form Relative (TFR) is defined as the difference between TFA in a particular time period and TFA in a previous time period. TFR enables a direct comparison between trainers with widely different absolute levels of form (TFAs). Now Nicky Henderson’s -3lb TFR can be compared with a trainer whose TFA is normally -12lb and is currently running at -9lb, to give a +3lb TFR.

For the analysis that follows TFR is calculated for each of the seven months October 2011 through to April 2012. TFRs are calculated by taking the TFA in each month by trainer and subtracting the TFA posted by trainer for the previous season 2010-11.  So, starting with October 2011, how is its TFR related to the TFRs posted one month later? Graph 3 below shows a significant relationship. At first sight this appears to be clear evidence that trainer form in October 2011 helps predict trainer form in November 2011.

Graph 3: Relationship between TFR October to November 2011  

What happens if we compare TFR in October 2011 with months further out? If form is temporary the relationship should decline through time. If October form predicts November form, it shouldn’t predict December form to the same degree. In the jargon we can postulate an  autoregressive AR(1) process. What we see in the data is the relationship shown in Graph 3 is as strong between October and November as it is between October and other months. See Table 1 below for correlation coefficients between t and t+1, t+2, t+3 using November 2011 as month t.

Correlation of November 2011 TFR with

December 2011             0.44

January 2012                0.51

February 2012              0.43

March 2012                    0.47

April 2012                       0.37

Table 1: Correlation of TFR months t with t+1, t+2, t+3 etc

We shouldn’t observe the same level of correlation across the months. It suggests form has a permanent component to it – an oxymoron. So how to explain this result? Imagine we can fast forward one year. We calculate TFA per trainer based upon the season 2011-12. This enables us to compare the results of a trainer in 2010-11 with the next season 2011-12. If we classify TFR as ABOVE or BELOW zero, and then further classify according to whether a trainer had a BETTER or WORSE season in 2011-12 relative to 2010-11, we see how RTF looks month by month in Table 2 below.

BETTER 2011-12 form                                  WORSE 2011-12 form

                              ABOVE       BELOW                                                            ABOVE       BELOW

ABOVE                     76               42                                                                         8                  28

BELOW                     33               16                                                                       29                109

Table 2: classification of form by month by year

Remember we have peaked into the future in calculating Table 2. It isn’t available until the end of the second season. The number of observations in the ABOVE-ABOVE and BELOW-BELOW categories are too high for the TFR measure to be useful as an indicator of form. In other words when a trainer is ABOVE or BELOW form in a particular month it is likely that they will continue in that category for the next month and the next month after that and for the duration of the season. The problem here is that the comparison used in the TFR calculation, namely the form of the trainer from the previous season, is likely to suffer from bias.  For some trainers it will be higher or lower than the true level of form a trainer can expect, and for those that posted towards the extremes of TFA they are likely to revert back to some degree in the next season. Bias such as this is difficult to remove and as a result relative measures of trainer form such as TFR are as flawed in their own way as absolute measures of trainer from such as TFA.

Summary

I started this analysis with the prior view that trainer form probably exists.  My view now is that it if it does exist it is very difficult to measure.  Absolute measures of trainer form do not exhibit enough variability, wheras relative measures have problems in deciding on an appropriate comparison. For some the measures of trainer form defined above might be too simplistic, arguing that more complex definitions are required. This is entirely possible. But as complexity of definition increases, particularly if it is one of many derivations tried, so does the risk that the measure will work only for the sample of data on which it was tried. If you torture the data for long enough it will tell you anything.

There is another possible use of trainer form – but more in how it is perceived by the market. Consider Graph 4 below. It shows TFA on the x-axis and Strike Rate on the y-axis by trainer for the 2010-11 season. It is the same data expressed in different ways.

Graph 4: Strike Rate vs trainer form

Strike Rate is a popular measure of form, RTF (and its variants TFA and TFR) less so. Yet Strike Rate is noisier and contains less information than RTF. Given the popularity of trainer form as an idea, and the popularity of Strike Rate as a proxy for trainer form, it is possible that the runners of trainers with a high/low Strike Rate relative to their TFA could have odds that are too far away from their correct values as the market considers these trainers, based upon a faulty premise, to be in or out of form. Armed with the appropriate data this is a testable proposition.

Introduction

The champion trainer for the season is decided using total prize money earned. This measure favours the very largest training yards, particularly those that have access to the offspring of top stallions.  As a result it is somewhat unsatisfactory measure of training yard success. Since Impact Values (IVs) correct for yard size by taking into account the number of runners as well as number of winners,  the playing field between yards of differing sizes is, to a good degree, made level when this measure is used. Whilst there are also limitations with using this measure across all races and for all trainers, the net is cast wider.  In this blog post IVs for different categories of race, namely maidens, handicaps and pattern races, are calculated, both raw and adjusted for Sire IVs (SA), then combined to produce a composite IV measure. Measuring IVs in different race categories enables a more complete picture of training yard success to be built. A by-product of the approach used is that trainers whose results are most and least influenced by the success of particular stallions can be identified.

Data & Methodology

The analysis that underpins this piece was carried out in the R statistical environment accessing Raceform Interactive data for the 2012 flat season. The R code is posted elsewhere for interested readers. To qualify for inclusion in the tables that follow, a training yard must have sent out at least 50 runners in handicaps and 100 runners in total over the course of the 2012 flat season, and be based in Great Britain (GB). A total of 139 yards met this criteria. These yards were then split into 2 groups according to how many different horses had been raced – 66 yards raced at least 40 different horses and are the focus of the analysis in this blog piece. The other 73 yards, smaller in size, were analysed separately and may be the subject of a further blog post. Since we know that on average larger yards deliver higher IVs than smaller yards (see my earlier blog post on this subject) smaller yards that perform well may not have appeared in the listings reported below and it is more appropriate to analyse their results separately.

Impact Values – Maidens

Maiden race IVs are likely to favour large yards with access to potential pattern class horses. Table 1 shows the top 10 yards ranked by sire adjusted IV in maidens. Raw IVs are also reported. Note the dominance of the Richard Hannon yard and the small difference between raw and sire adjusted IVs compared with the larger differences between IVs for Saeed bin Suroor and William Haggas. The large number of horses at the Hannon’s yard appears to confer a substantial advantage in being able to place horses to good effect within maidens. The same comments apply to Richard Fahey’s results. In both yards the large number of horses at their disposal appears to outweigh any advantage given to other yards via ostensibly better bred horses.

Table 1: Top 10 training yards by Sire adjusted IV in maidens

Impact Values – Handicaps

Table 2 shows the top 10 yards ranked by Sire adjusted IV in handicaps. Raw IVs are also reported. Sir Mark Prescott Bt tops the table, although in common with the majority of the trainers in the top 10 his Sire adjusted IV is substantially lower than his raw IV. Noteworthy are the results of Chris Wall and Michael Appleby, whose IVs are hardly affected by the relative success of the sires of their horses in training. Part of this result is due to their lack of relative success in maidens, suggesting their horses are likely to be highly competitive when they move out of maidens  into handicap company – Chris Wall’s IV in maidens was 0.43, whilst Michael Appelby sent out no maiden winners in 2012. In contrast Sir Mark Prescott Bt, along with 6 other trainers, delivered IVs above 1 in both maiden and handicap company. The other 6 were Marcus Tregoning, Luca Cumani, Sir Michael Stoute, Ed Dunlop, James Fanshawe, Roger Varian and Mick Channon.

Table 2: Top 10 training yards by Sire adjusted IV in handicaps

Impact Values – Pattern Races

Table 3 shows the top 20 yards ranked by Sire adjusted IV in pattern races. Raw IVs are also reported. The results are more difficult to interpret than maidens and handicaps for individual trainers because of small sample sizes. The Richard Hannon and John Gosden yards dominate the table in terms of number of winners and runners, however the Sire adjusted IVs for both trainers are noticeably  lower than their raw IVs. It is possible this  result is an artefact created by their substantial relative success in producing pattern class winners during the 2012 flat season. A number of yards that perform well on the IV measure in maiden company do not appear in the table below.

Table 3: Top 10 training yards by Sire adjusted IV in pattern races

Impact Values – Composite Measure

A composite IV is calculated by combining together the IVs for maidens, handicaps and pattern races by trainer, weighting by the proportion of runs that each trainer had in each category.  Thus a trainer without runners in pattern races would not be penalised for his non-participation, and the biggest contributor to each trainer’s IV is from the category of race in which they had the biggest proportion of runners. The composite measure was also adjusted for Sire IV. Using this measure Sir Mark Prescott Bt was the top trainer on the flat in 2012, followed by William Haggas and Marcus Tregoning. Noteworthy results were produced by Henry Candy, David Barron, Michael Appleby and Chris Wall, each of whom saw their IV increase after taking the Sire IV adjustment  into account. For 16 of the 20  trainers we see the opposite, suggesting that the adjustment for bloodstock quality used here via a Sire adjusted IV does not go far enough. I will return to this subject in another blog article. Thanks to Declan Meagher and others for making this point  on the separate blog post “Do Small Training Yards Punch Above Their Weight?’.

Table 4: Top 20 training yards by composite IV adjusted for Sire

Training Yards Success & Relationship with Sire Quality

How many training yards are able to deliver improved IVs after the Sire adjustment is taken into account? Remember for successful yards the natural direction for the Sire adjustment to take your IV is downwards. This is because the better quality Sires make an outsized contribution in terms of siring winners. So the yards that are able to increase their IVs after this adjustment is applied are worthy of note. There are 10 yards out of the 66 – see Table 5 below –  that were able to deliver an adjusted composite IV both greater than 1 and higher than their raw composite IV. Henry Candy and David Barron’s results are noteworthy.

Table 5: Top 10 trainers with improved IVs after Sire adjustment ranked on Sire adjusted IV

What of yards that see falls in their IVs after the Sire adjustment is applied? Table 6 ranks the 10 training yards most affected by the Sire IV adjustment. These yards are still highly successful – they still post IVs substantially greater than 1. However, using this metric suggests that these training yards are more reliant than others on the quality of their bloodstock for their success.

Table 6: Bottom 10 trainers with reduced IVs after Sire adjustment

Summary

In this paper the criteria used for measuring training yard success is a Sire Adjusted Impact Value derived from results delivered in maidens, handicaps and pattern races. Using this measure Sir Mark Prescott Bt was the top trainer on the flat in 2012. It is probable the Sire IV adjustment used does not go far enough in terms of correcting for quality and another blog post will address this point.  A small number of trainers produce IVs that improve after an adjustment for Sire quality is made. These training yards are of particular interest.  .