Quantifying Match-Ups

Critical to avoid small sample sizes of data...

Yesterday I saw a post on Twitter that I found interesting enough to engage with, focusing on the head-to-head records of two players in the Women’s Premier League.

I’m pretty big on match-ups and find them an integral part of cricket, no matter the format. It’s not far off a chess game which needs to be solved as optimally as possible, but very rarely do I look at individual batter v bowler head-to-head data.

This post reminded me of what I wrote in my e-book Strategies for Success in the Indian Premier League (written in lockdown 2020 - email here if you would like to purchase a copy) and my views have shown little change from then until now on this subject.

Here’s an abridged version of my chapter on match-ups…

How many times have you seen, either on TV or on social media, a player’s record against a particular player discussed? As access to data becomes more available, this type of stat is becoming more and more commonplace, with perhaps a stat shown that ‘batsman x has faced 50 balls from bowler y, scoring 80 runs and being dismissed on five occasions’.

Often, this is then followed up by some ‘analysis’, but the problem with this is multi-fold. Looking at the example above, several narratives can be driven - firstly, some may wish to make the point that the batsman has a match-up issue with the bowler (because he averages 16, and has been dismissed every 10 balls), but those who are looking for positive spin might focus on that batsman having struck at a strike rate of 160.

So, which is true? I would venture to suggest that in the above example, we may as well toss a coin. There is no way that any conclusions can be drawn, partly because of the conflicting narratives above, but also because context and sample size are both lacking. For example, this stat might be really quite acceptable for a batsman who typically bats around number eight and who generally has a job of hitting a few balls in the death overs, but if it was for a top order batsman, it’s far less acceptable - so we need more context. In addition, the sample size is minuscule - as I referenced in the discussion during the introduction to this book. We have no idea, or confidence, as to the likelihood of the results of this sample being replicated in the future. If we benefited from being able to add a zero on the sample size, then we’d have more confidence, but still not as much confidence as if we had a sample size of 5,000 balls instead.

The problem is that in T20 cricket, when it comes to batsman vs bowler match-ups, small sample sizes reign. In the course of writing this chapter, in addition to my previous work, I looked numerous batsman vs bowler match-ups, and even found it difficult to find many where the sample size was over 200 balls in T20. It was even tough to find many over 100 balls. So, if an analyst suggests certain match-ups to coaches or players because of individual batsman vs bowler data, it’s virtually guaranteed to be of a tiny sample size, and therefore, at best, potentially misleading. If you’re a coach and your analyst offers this type of analysis to you, make sure you ask them how confident they are that such a ‘trend’ is likely to continue...

A major issue with regards to small sample sizes in particular would be the impact on batting averages - much more than strike rates or boundary percentages. I’ll re-iterate a paragraph I wrote in the introduction here:-

‘Averages have the propensity to mislead more in instances of small sample sizes than batting strike rates or boundary percentages - imagine a player scores 200 runs from a 150 ball sample against slow left-arm spinners, being dismissed five times. They’d average 40, but even if a few shots that they played ended up differently - for example, an edge past the stumps hit the stumps or a few catches were taken instead of being dropped, it wouldn’t be difficult to envisage a scenario where they could have averaged in the low 20s instead, almost half their current average from that limited sample size. It is very difficult to make the same observation for batting strike rates or boundary percentages - let’s say a few boundaries ended up being singles or twos, even at such a small sample size, it wouldn’t have nearly as much effect on boundary percentages and therefore strike rates as the above example with batting averages.’

Given that there are extreme sample size challenges with individual batsmen vs bowler match-ups, perhaps a lot of readers might be thinking that the next best approach might be to look at individual batsman match-ups against particular bowler types, such as, for example, how they perform against left-arm pace. However, even this can be fraught with difficulties with regards to being able to draw reasonably accurate conclusions.

For a start, certain bowler types are rarer than others. A lower proportion of left-arm pace is bowled compared to right-arm pace. Similarly, left-arm off-spin tends to be utilised less than right-arm off-spin, and particularly in the Indian Premier League, compared to right- arm left-spin. Left-arm wrist-spin, exhibited by Kuldeep Yadav, for example, is an even rarer bowler type, although I have noticed that a number of young bowlers with high potential from Afghanistan bowl via this style, and this could be growing bowling type.

In addition, it is important to consider the context of the match-up from a player ability perspective. To give an extreme example, let’s think about a hypothetical match-up between a 22 year old top order batsman and a 34 year old bowler, who had several battles in a T20 league four years ago. The problem with considering any data from a historical match-up is that ideally, the ability levels of the two players from the historical data should not hugely deviate from their current ability levels. However, in this historical example, we find a 22 year old batsman who is four years along the age curve from the previous meeting (with a strong likelihood of him improving since the historical data was generated) facing a 34 year old bowler who previously was around peak age but now has the potential to be in decline. While not every match-up will have irrelevant historical data due to the subsequent direction of the careers of the two players, it’s certainly something to consider.

Furthermore, we also must take into account whether the time-frame for match-up data is relevant for the current match-up. For example, how long a time-frame should be used when looking at the match-up data? Scoring rates in the Indian Premier League have increased through the years, and in particular, in the last three editions (2017 to 2019) at the time of writing. If a batsman scored at a strike rate of 120 in the early years, it might be quite acceptable - while potentially being much less acceptable currently. Given this, should adjustment factors be used? In any case, probably keeping relevant historical data to a three-year time period would be quite worthwhile, but yet again, this creates issues with sample sizes.

If it wasn’t already difficult enough to find examples of big sample sizes of data for batsman vs bowler, or batsman vs bowler type match-ups, there are even more factors which should be considered, and one of these would be potential deviation in conditions. To use a tennis context, why would you look at head-to-head results on clay courts if an upcoming match is to be played on grass, when the skill-sets required to succeed on the two different surfaces (the two surfaces which generally deviate the most in terms of court speed on the professional tour) is completely different.

This can be applied to cricket as well. How would your perception change of the example given earlier if the 50 balls faced by a batsman with 80 runs scored but five dismissals were delivered by a spinner in the Bangladesh Premier League (a league which often offers very spin-friendly conditions), yet the upcoming match-up between the same players was to be in the T20 Blast - where conditions are generally much more friendly to batsmen, and less friendly towards spinners? It might not be too far to suggest that the historical data is, again, completely irrelevant.

Perhaps one way to obtain bigger sample sizes which could be slightly more reliable would be to look at a batsman’s record against both pace and spin bowling in the last three years - as opposed to the individual variants of each type of bowling, and then look at this in conjunction with how either right-hand or left-hand batsmen (whichever is relevant) generally perform against a specific bowling type (e.g. left-arm pace) during that time frame, in comparison to how right or left-handers perform against other bowling types.

While not perfect, focusing simply on a batsman’s record against pace or spin bowling for example, would generate a more robust sample from a size perspective, while having time relevance and also giving insight into how similar batsmen in general perform against that individual type of bowling, and in an imperfect world, this is an approach I quite like.

Concluding this chapter, despite being able to give some thoughts and ideas about how to look at the match-up problem, it’s fair to say that the entire concept of match-ups is extremely difficult. In my view, there is extreme scope for small samples of data in this area to, at best, mislead, and at worst, be extremely dangerous.

Decision-makers at teams need to be very aware of these difficulties and not be seduced by drawing conclusions from historical data with small sample sizes and with little relevance to the current match-up.