The stats that back Cheika getting the sack - Green and Gold Rugby
Analysis

The stats that back Cheika getting the sack

The stats that back Cheika getting the sack

I haven’t come across too many people making a strong stand for Michael Cheika to remain on as Wallaby coach since the team’s departure from the Rugby World Cup last weekend.

But there is a level of thought out there that the disappointing results of the last few years was just a continuation of a trend that commenced basically after we won our last World Cup way back in 1999. Ben Darwin, through his work with Gain Line Analytics has proposed as much for some time, whenever talk of a new Wallaby coach has been raised.

Now Ben is a former Wallaby, a former top level coach and has made a living out of statistical analysis of team performances.  I, on the other hand, have not.  But I have done a nice pretty graph. I’m no statistician but I think it tells a story.

I mentioned this on our podcast , but here it is in a visual form.  What I’ve done is broken down Wallaby team win percentages into 5 year increments. Now there is some latitude taken with this as in the early years as, for instance, between 1899 and 1903 we only played 5 tests. And then none following 1914 until 1920, and again none from 1938 to 1946. But you get the idea.

Wallaby Graph

The Wallaby overall win percentage is just a smidge over 50% from my records (including games played by the Waratahs early last century, matches that were later deemed to be capped matches).

Delving into this a little more deeply, from my perspective Australian Rugby can be split two phases – pre 1975 and post 1975. It’s actually more specifically around 1978 as a turning point, but as a 5 year increment it’s 1976-1980 that becomes central to the upturn in Australian rugby performances.

Put simply, we were not a high performing team before this. We had a win record of just under 30%.

From that 1975 period onward, the record in just over 60%. Now obviously we had a particularly successful decade in the 90s when our win record was a over 75%, but the trend upwards started in the late 70s.

The upturn came as a result of a few endeavors, by all reports, including the appointment of Dick Marks as a National Coaching Director, the advent of the Australian Institute of Sport (after the debacle of the 1976 Olympics) and the emergence of that brilliant 1977/1978 Australian Schoolboys team. There are more, I am sure, but these are common reasons provided.

But back to the graph. It shows win percentage by 5 year increments. The red solid line shows the overall Wallaby success rate, the green dashed line shows the success rate until 1975 and the yellow dashed line shows it post 1975.

"look at me, I am the captain now"

No one is happy about the Wallaby win rate.

The theory that we’ve seen a declining performance line seems inaccurate in this graph (acknowledging Darwin is likely using way more in-depth information that I am). Regardless, if you agree that our success rate in the last four and half decades is about 60% (not bad for a country our size with the competition from other codes), then the win rate of the last four years of less than 45% stands out as a massive under performance. In fact the win percentage over this period is the lowest it’s been since that 1971-1975 period when we inherited the “woeful wallabies” tag.

Now can we turn it around? I bloody well hope so as I don’t think I can handle another RWC cycle of poor performances.  If we are to turn things around then perhaps we can look back to that critical moment back in the mid to late 70s.  It may not be another Dick Marks type appointment or next iteration of the AIS.  But it will take a commitment to looking at the approach to rugby coaching differently to how it has been done in the past.  A focus on intelligence and being at the leading edge of coaching and our approach to the development of the support in this country.

Junior Wallabies are 2019 Oceania U20 Champions

Junior Wallabies are 2019 Oceania U20 Champions

Because, just as they had with that 77/78 schoolboy team that brought us names such as Ella, O’Connor, Ella, Hawker, Darcy, Ella, Melrose and Roche, we have another generation of young stars coming through.  Recent success by the Australian Schoolboy and Under 18 team, as well as the Under 20s provides us another potentially rich vein of talent.  We need to ensure that the structure and support is in place to ensure the names of McReight, Wilson, Lucas, Harrison, Bell, Nawaganitawase, Pasitoa, Roach, Flook, Pollard, Gordon et al reach the levels their potential has them positioned for.

 

  • Cmac

    Good analysis, Reg, I like the way you’ve visualised the data.
    (and although you claim not to be a statiscian – good data visualisation is an important first step in statisical analysis – props to you).
    The most recent 5-year block does look way down on the post-1975 trend… but it could be a blip (it’s only one data point). One thing you could do with that data is ‘chunk’ it by 1, 2 or 3 year year intervals and see if you get a qualitatively similar pattern. Some whisker bars that indicate variation around the mean win rate would also be interesting.

    • RugbyReg

      interesting. Will have a look at that to see how it comes up.

      Cheers

    • Parker

      A whisker bar sounds like the place you’d find Hoss having a single malt and smiling like a Cheshire Cat.

      • Nutta

        When you find that bar call me. I’ll have a slurp.

    • dsb

      Quantitative

    • Mike D

      Are there enough tests in any given year to provide a statistically valid data set? Especially given we play the ABs 3 times per year (Usually, this year is the anomaly) and we should normalise our win/loss ratio against the rankings of the teams played. By chunking in 5 year increments that helps the signal to noise ratio so don’t need to worry about normalising perhaps.

  • Nice work Reg.

    One of the things that would be interesting to see would be something like the AB and the SA records over the same periods for the same thing: win percentage in five year chunks. Why those two teams? They have, at least in the last 20 years or so, the same playing patterns as the Wallabies, so if there’s a trend that’s affecting SH sides you’d expect it to show.

    Another thing to remember is that different stats show different things. Both you and Ben could be right. Ben’s stat is roughly about how effectively the Wallabies generate go forward. That approximately correlates to winning – generally the side that does better at it, wins – whereas you’re measuring directly wins and losses. But you can’t really coach wins and losses directly (every coach tries to coach for a win you assume but saying “lets win this one” isn’t a good plan), but you can coach ways to repeatedly have gain line success, at least to do better at it. That should then lead to better performance in the wins and losses column (at least in theory).

    • Patrick

      Ben’s stats don’t allow for the fact that Chieka has had us generating as much go-forward in our own 22 as in the opposition’s, which we did not historically bother trying to do because we un-Australian-ly kicked it the hell out of there.

      • Yowie

        Shame on those prior generations of Wallabies for being so un-Australian.

      • I don’t know the fine details of Ben’s system. A lot of these apparently simple systems hide a weighting factor to avoid just such an issue. For example in your own 22 you get x0.5, between the 22’s x1 and in their 22 x2 for your gain line successes.

        This way running it from deep is rewarded if it works but you get more rewards for playing in the right bits of the park.

        Obviously there are more zones on the pitch you could use and ways to weight the score: you could weight for n-1 where n is the number of successive phases crossing the gain line, and metres across the gain line too. If the theory is multiple gain line successes in a row disorders the defence, keeping on going is better… and bigger penetrations are more likely to make it break, no?

        Like I said, I don’t know if Ben does any of these, and I don’t know how well the ones I’ve suggested model to outcomes, but they’d be where I’d start and I’d be surprised if something like this wasn’t in the mix tbh.

        • Patrick

          I was mainly being snide. I do expect it is somehow taken into account.

      • Who?

        I would suspect that Ben’s stats may also fail to account for the fact that the 9/10 combination in the last game had played literally 20 minutes together through the pool matches, and hadn’t played together before this year for what, five or six years?!
        And let’s not forget, it was Ben’s analysis last year strongly advocating for picking combinations that drove about six months of discussion. It was THAT critical to ensure players had combination.
        .
        I must say, I’m not as sold on combination as Ben. I think it’s important, but I also think that coached game plans and strategies can fast track and (to a small extent) compensate for combination. Thought I don’t believe Cheika’s strategies necessarily helped with that. Just going off the regularly missed passes through the RWC, particularly in the early games.
        .
        So, if Ben’s adamant we pick players with combination, why would he justify the retention of a coach whose MO was to constantly shuffle his sides?

        • Custard Taht

          In defence of Cheika, he did try and stick with a 9/10 combination, being Genia and Foley.

          Problem was, it was shit, and if not for the selection panel, that would have been the combination he would have used as his primary for the RWC.

        • Who?

          He stuck with a combination for 3 years, then changed it for the RWC. That’s a Robbie Deans 2011 move right there. Except that, when Robbie changed things up going into a RWC year, he put McCabe at 12 and stuck with him, excepting for injuries. Cheika didn’t do that with Leali’ifano this year, and he swapped Genia and White every game.
          .
          He also swapped our centres constantly this year. And, in the RWC, the loose forwards, too.
          .
          I guess we’ll never be certain how much of the changing was Cheika and how much was the selection panel. But if there’s a time to have a set combination, it’s the RWC. You can shuffle things other years.

    • Parker

      Isn’t the objective to convert go forward into points on the board. So tell me, which statistical approach is more indicative of more points. All that going forward and failing before the line is futile.

      • Yes. But the stat we most often see of “metres made” is really pointless. If the other side kick deep and chase in a slow but solid line, you can get a lot of metres, then just grind to a halt. Old indicators like possession and territory are meaningless when you can tackle for most of the match, score off turnovers and bad kicks in a phase of two and win – and not only if you’re the AB these days.

        Gain line success attempts to fairly simply measure attempts to bend or break the defensive line. That doesn’t ultimately guarantee you score of course, but we’ve all seen players over the line, unopposed and drop the ball – not even being open over the try line can actually guarantee scoring a try before you ground it safely!

  • Seb V

    Nice work, but I really don’t see the point in analysis dating back to the 70’s. Does comparing an amateur sport really compare well to professional?

    • formerflanker

      Good point, but some ancient followers of the game (cough) still retain wonderful memories of winning rugby and scintillating skills. Sure the stats tell one story but the emotion doesn’t differentiate between amateur and professional eras.

    • RugbyReg

      if only to show how far we’ve fallen. We’ve been batting at a 60% average since the mid-70s. The fact is, our win record in the professional era is still 60%.

      A 45% win record over 4 years is poor regardless of the era

    • Interesting point.

      For some stats I think you’ve got a strong case. For example, you might reasonably expect full time professionals to have better tackle completion, better fitness (and so a higher average speed over the match) etc.

      Rules changes will affect number of turnovers won/lost, penalty count, red and yellow cards (even the existence of cards) etc.

      But, unless there’s a dramatic change in one direction for one group of tier 1 countries in win% and in another for other tier 1 countries in 1995 (when the sport when pro) you’d imagine win% should remain broadly unchanged. After all, whatever effects there are on how the Wallabies play, also affect the other tier 1 countries as they adjust to the professional era.

      I’ve said tier 1 in there deliberately. Japan to the contrary, since ‘95, wins by tier 2 countries over tier 1 countries have become rare as the gulf with having a fully professional home league at least one tier deep (SR, Pro14, Top14 etc.) typically makes a difference. Italy may come out of the doldrums, Japan really should be moved up to tier 1 IMO, and there are some countries like Georgia and Russia that seem to be knocking at the door, at least for equality with Italy.

      So I think, while it’s worth thinking about, for win% you probably can compare back over the eras. Style of play, tactics, tackle completion and a load of other things, not so much, but because the change to professionalism should affect the opposition equally, it shouldn’t have an effect on win%.

  • Flavio

    We also suffer from playing the AB’s 2 to 3 times a year. No other nation has that so would be interesting if we took the Bledisloe out of the equation

    • Patrick

      Yes but this was also the case in our most succesful periods as well so hard to see the point, this is comparing us to us not us to other countries.

    • The SA throughout the tri-nations and RC era play the AB more or less as often.

      There is an argument for seeing Australia and SA since ‘96 with and without matches against NZ. Since ‘96 is conveniently the start of a RWC cycle, 4-year buckets so you can see them over RWC cycles would be nice too. I might try and sort that out.

      • HK Red

        We play 1 more game against them every year.

        • To be pedantic, only in most years. I started on the win% in 4 year cycles I talked about. In 96 I think it was (the years are a bit of a blur now) SA played NZ in the nascent Tri-Nations home and away, then a 4-test series against them for a total of 6 matches, winning 1 and losing 5. Ouch.

          In recent years you’re right though.

  • Patrick

    Agree with all that you have written, and I appreciate your taking the time to present the analysis as well!

    I think over and above the coaching and performance structures we need to overhaul the governance structure.

  • Bobas

    “The stats that back Cheika getting the sack”

    But he didn’t, he is choosing not the reapply for the job.

    • Only a matter of timing.

    • Yowie

      But there is less Dr Seuss poetry to “The stats that back Cheika choosing to not reapply for the job”

      • sambo6

        The coach was a stubborn old Cat
        He didn’t want to kick it back
        It left his team on the rack
        So the cat got the sack
        and hopefully wont come back

        • Yowie

          When Raelene and Cheika
          Battle with paddles at a function
          about Scottie sending players
          and Turinu intervenes in their dysfunction
          They call it a Tweetle Beetle
          bottle puddle paddle battle muddle

        • Mike D

          Thank you Mr Socks.

    • RugbyReg

      worthwhile contribution. Thanks.

      As the headline suggests, these stats back him getting sacked. Not him not reapplying.

      • D. Braithwaite’s The Brumbies

        It frustrates me a bit that he’s trying to create this narrative of him having done a pretty good job, but choosing not to reapply due to falling at the last hurdle.

        Not that I expect him to come out and say ‘I was a terrible coach, the worst for 40 years, who wasted the careers of plenty of good quality players, ended plenty of others, took the wallabies from a powerhouse to a joke and coached all the intelligence out of the team with my brain dead game plan’ (despite all this being true).

        At the very least it would be nice to get an admission that his game plan these last 6 years didn’t work and that results these last 4 years haven’t been good enough.

        Most of all, however, I am just relieved that he is gone. What a terrible time these last 4 years have been.

      • Bobas

        Sacked is made redundant or terminated. Cheika gets to skip away thinking he did he job.

        • RugbyReg

          Yes. I know what sacked is thanks for being patronising. I was suggesting the stats indicate he should have been sacked.

        • Bobas

          I wasn’t meaning to be patronising, the headline suggests to me that he’s been sacked and this is why.

          We are in total agreement on the facts, stats & suggestions publicised.

  • Jason

    I’d love to see/run the numbers that consider the strength of opposition (perhaps the World Rugby Rankings Points) and tracks our performance vs oppositions better than, equal to, and worse than us. Anecdotally, I think it’s pretty obvious we’ve never been playing worse oppositions (overall) and performing worse against them.

    • Patrick

      We have that, it is the world rankings.

      • Jason

        Well that accounts for other teams wins losses too. But I’m more thinking like ‘we win against teams within x points y% of the time, but lose to teams z points above us, and always beat teams w points blow us.’

  • Cornchips

    Why was it 5 years? Why not 4 years to account for only one world cup where we blow 3 smaller teams off the park

    • RugbyReg

      to split the decade equally. And to take into account the 90 odd years there wasn’t World Cups

  • idiot savant

    Thanks Reg. This further supports the intervention of RA. Yes it was way too late. But I don’t think anyone has yet made clear the depth of support for Cheika amongst former and current Wallabies, RA board, RA members, and the public. He is a favorite son of NSW. No ceo could have moved against him and survived. 9 losses out of 13 wasn’t enough for anyone to sack him and survive. So it had to be a patch with Johnson and the selectors. That in itself was brave on Raelenes part and many of the calls for her sacking now are really anger that she dared to get in Cheikas way. Many will dispute and discredit your evidence. We are truly living in Trumpian times.

  • Pablo Glasheen

    I think Chieka could have used his famous names of Genia, Beale and Pocock off the bench. Other younger players were in form and playing better rugby. His famous names didnt play to their former standards and made mistakes at crucial stages of the game. The new guard will now step in but I would have liked to see them given the chance against England.

  • AllyOz

    Not sure I am allowed to even mention it but Rothmans put a lot of resources into all forms of sports coaching and development in the 1970s & into the 1980s. My dad said the cricket coaching course that he did was excellent, focusing on key skills and fundamentals. Whilst we don’t want to be handing out packs of fags to our under 8s it would be good if there was a corporate sponsor looking to specifically donate to development and coaching of the game. Do Maccas have some kind of tie to this with Auskick? Rod Kafer did some work on coaching systems but his position is now gone. Perhaps once Scott Johnson has reviewed and renewed the Wallabies coaching staff he will start to concentrate on more of these sorts of issues. I think in Scotland part of his brief extended into the second tier and general coaching.

  • Nutta

    Here’s to you Reg for some cerebral nous. Grand.

    I find it illuminating to go back and listen to interviews with the likes of Dave Brockoff, Finnane and Cornelson et al from the ‘step forward’ period of the later 1970’s. After the humiliations of the early 70’s including the seminal loss to Tonga, a group of fair-dinkum people finally committed to changing what we did to suit the landscape. We had talent – we always had talent – but they committed to address their weaknesses. Our weaknesses to that point were in general physicality particularly at the set-piece. Lesson – identified the weakness and fixed it leading to period of success.

    Then came the Grand Slam where again the set-up at the time largely driven by Jones (love him or hate him, historical fact is just that – historical fact) focussed on addressing the structural weakness. So the game-day weaknesses were identified (lineout and scrum) and the cultural weaknesses were likewise identified (shite training and fitness) and the work was put in to get the right players (Topo, Andy McIntyre, Steve Cutler etc), training with purpose and playing the right game-plan to beat that opponent. Lesson – identified the weakness and fixed it leading to period of success.

    Get to the early 90’s and Dwyer – backed by a supportive ARU – did it again by being sure we had a lineout-useful backrow (ie Coker over Miller) and a serious scrum (Daly/Kearns/Link/McCall). Lesson – identified the weakness and fixed it leading to period of success.

    Get to the late 90’s and Rod McQueen & Co does the same again – identify the weakness and structurally address them.

    The lesson is in identifying where we are weak and making a fundamental change to address it.

    Now we part-accomplished that after 2007/2011 by putting in serious corrective actions to rebuild our Tight5 stocks to the point where we could now field 2 complete Tight5’s able to take on any pack in the world and I’ve never known Oz rugby to ever have that depth.

    But our challenges are now different. Our challenges are now more holistic and are around game-smarts (which has been willfully trained out of a generation of players being slaves to ‘the process’ and coaches who are extremely sensitive to preemptive engineering out any notion of player-independence) and also about producing the sheer production line of players required to cover the demand inclusive of the siphoning impact of overseas dollars and competing codes. It’s different but it’s the same. It’s about recognising the weakness we have and addressing it. And again it’s going to take a whole-of-game approach to address it.

    The question is do we have the senior leadership in-place at RA level to face-up, own it and take it on?

    • idiot savant

      You make some great points Nutta. We don’t need a whole of game approach to fix it tho, just a new coach. We have great forwards as you point out. All we need is the basics to improve. Exit 22 by kicking. Kick deep to corner on restarts and get ball back. Run straight and recycle quickly before going wide. We have the players to do this. Any school boy first XV coach will do.

      • Nutta

        I don’t think it’s quite that simple though.

        Re-educating existing players to think again instead of blindly following a formula after a decade or more of continuous reinforcement and conditioning is not simple.

        Giving coaches and selectors at grass-roots the confidence and tool-kit to facilitate smarter rugby will require significant investment and support currently not there (so from where and paid for how?)

        Giving big clubs and competitions reason to act in the best interests of THE GAME and not themselves requires radical re-thinking and realignment of administrative approaches.

        And unlocking and engaging the sheer numbers required to produce the player-base needed to fill the pipe and compete again will require a huge cultural shift out of our own snobbery and nepotism to a much more professional, commercially-oordinated and above-all an inclusionist orientation.

        But none of that is impossible.

  • I’m not sure if Disqus will let me do this, but hopefully a table of Australian and South African results in the Tri-Nations era, by RWC cycle. Foolishly I’ve missed counting draws so the win percentages are a bit higher than they should be. Generally that’s like 0-1% high and I figured meh, but in the last cycle fort the Wallabies it’s more than that. I calculated it for that one and included it in brackets.

    Note that, counter-intuitively, a bigger difference between the “all” column and the “excluding AB” column represents a WORSE set of results against the AB. If you want to test it, imagine you play 60 matches and win 40, that gives you a 67% win ratio. If 10 are against the AB and you win all 10… excluding them would mean you played 50 matches and won 30, a 60% win ratio, if you lost all 10, you played 50 matches and won 40 still, an 80% win ratio. If you won 5, lost 5, you played 50 matches and won 35, a 70% win ratio. So the better you do against the AB, the WORSE the effect on your post-exclusion ratio.

    Also note that the Bokkes are still playing, so their last cycle stats will change a bit.

    All games (except draws)
    Excl. AB games

    RWC cycle
    Australia
    South Africa
    Australia
    South Africa

    96 to 9974738383

    00 to 0367517161

    04 to 0765747482

    08 to 1163617166

    12 to 1559717080

    16 to 1951 (48)556263

    It’s noticeable that there’s about a 10% penalty in each cycle from playing the All Blacks… ouch. Even in good cycles, SA won 71% of their games in one cycle, Australia 74% in another, and if they hadn’t played the ABs they’d have done 9% better!

    South Africa show some ups and downs. There’s a high of 74% in the cycle in which they won the RWC, and one of 73% in the cycle in which Australia won the RWC. There are a couple of lows in the low 50’s too. Without other looking at the other tier 1 teams this is what you might guess at, some highs followed by crashes as the team rebuilds following retirement of star players. I would expect you’d see, with different phases, a lot of tier 1 nations show this sort of peaks and troughs pattern. The win-percentages might vary, but you’d expect the pattern to be there.

    Australia, though, show a different pattern. The highs of the 99 RWC win lead to a drop off, and then there’s a plateau – 67, 65 and 63% win ratios are the same to the margin of error – a game here or there could even those up easily and they’re 71%, 74% and 71% if you exclude the AB suggesting it’s odd results against the ABs that make the biggest difference. But after 2011 the results have fallen off, and in the last cycle under Cheika there’s an 11% decline in the win rate (including draws).

    People who want to point the finger at Deans – he handled the end of that plateau in essence. I’m not saying he should have been kept, but these stats suggest things have fallen off since him, not under him. And in Cheika’s full cycle it’s been terrible. Reg’s original thesis is strongly supported by breaking it down this way.

    There’s not evidence here to suggest why – you can try and point to your favourite theory but some things to consider.

    This says nothing about the systems around the Wallabies. Has SR stopped producing the talent and/or the combinations to compete at the highest level? That could account for this decline, and certainly the last cycle has seen a fair few changes to the SR season.
    Is there an issue of poor selection and/or poor tactics imposed on the players? The “it’s all Cheika’s fault” solution.
    Is there an issue in the elite levels (and possibly lower) of Australian rugby? We hear a lot about the desire of players in other teams to work on their skills, to keep improving and so on. We can see players return to Australia and apparently lose skills they showed playing overseas. Why? On the flip side, we rarely see players improve, even when there are weaknesses. Koroibete is a recent exception. But that we can name the exceptions is troubling.
    Is there something else? Is the wider production line broken and that’s only just coming to light perhaps?

    This stat can’t answer that question on its own. But it can shine some light on it and give you some ammunition to think about your answers.

  • Hoppy

    I have been as frustrated as any of you with the wilful insistence of Cheika on his way. Despite his protests that he is always willing to change and learn from others and from his own mistakes, he has done exactly the opposite. On top of that he has demonstrated an abysmal level of emotional intelligence in his comments and behaviour since the qtr final loss to England.
    However, as others have said on this blog the malaise in our recent rugby performances is not just down to Michael Cheika.
    The AB’s were quicker and smarter to recognise and correct their own malaise after the period up to 2004 – just read LEGACY by Kerr. That review and the continually evolving action plans that have resulted to improve their holistic team performance since that time have resulted in an astonishingly successful team at all levels – not just on the pitch. It required the eschewing by all involved of any ‘blame games’, a willingness on the part of those in charge to admit they didn’t have the answers and to accept that others did, and an acceptance that there were capability gaps in leadership, coaching, support, administration, life and rugby skills – and the list goes on.
    RA could do a lot worse than adopt and apply the Legacy approach as a framework for rejuvenating and improving the performances (and outcomes) of the Wallabies and all our other elite rugby teams through the next RWC cycle and beyond. The details in some of our lessons might vary from theirs but a lot will be eerily similar.

  • Andy

    Great article Reg.

    So good coming to a website and reading articles and hearing podcasts that are
    purely objective and use stats and facts to back up their positions.

    I just finished listening to the Drew Mitchell and Hoiles podcast and all I can say is that I just wasted 45 minutes of my life with the complete load dribble they spurt. Couldn’t give an unbiased opinion if they tried. If you listen to them it’s like the Cheika era was overly positive for rugby in this country. And to make it worse the way they talk down their listeners if they get asked a hard/serious question, really just leaves a sour taste

Analysis
@RugbyReg

The original prop in a prop's body, but thankfully I have the rugby mind of a prop as well.

More in Analysis