One NRL bunker official achieved a 98.5% success rate - this is how he did it

Rugby league fans love to blame the bunker when things don’t go their own way - but these are the facts that prove the bunker barely ever gets it wrong.

You walk into Andrew Abdo’s office at NRL headquarters in Moore Park to find spreadsheets on every referee and bunker official in the game.

Column after column of numbers and detail on every second of every game on every millimetre of turf.

Not just tries, forward passes, knock-ons and foul play.

We’re also talking play-the-ball speeds, 10-metre compliance, ball-in-play times and the bunker decision times, the adjudication process and accuracy.

Welcome inside the refereeing engine room where the likes of Ashley Klein, Gerard Sutton, Adam Gee and Grant Atkins have their weekend performances as heavily scrutinised as any player in the competition via a sophisticated scoring system.

The captured data is fed into a computer algorithm, along with video observations from a referee’s coach, to determine a KPI score for each referee.

For the first time the NRL has opened its doors to reveal how referees are assessed – and why Klein retained his position for the State of Origin decider.

That he was not judged on a 9-0 penalty count to Queensland in the opening 45 minutes but on a detailed analysis and form from the last month of the competition that shows he’s still the No.1 referee in the game ahead of Atkins and Gee.

Revealed: The referee KPI determined by a sophisticated scoring system

HOW IT STARTED

In 2020 the NRL was under attack about the quality of the football and refereeing inconsistency.

Old Channel 9 boss Hugh Marks launched a scathing attack on the NRL over the product.

Rule changes were introduced to speed up the game and the independent commission identified the need to introduce specialised KPIs to make referees more accountable.

To improve the game as a spectacle, the referees had an important role to play.

They would be judged and held responsible for the play-the-ball speed and ball-in-play times.

All those incidents that send social media into meltdown during games are now carefully recorded.

“The aim was for a faster and more free-flowing game,” Abdo said, “And that’s what we’re now achieving. A couple of years ago ball-in-play was about 54 minutes. We are now pushing 57 minutes.

“That gives us continuity in a game, fatigue then creates space to play football. The referees policing the key rules around the ruck speed and 10m compliance are so important.”

In comparison to the NRL, a rugby union game has around 30 minutes of ball in play.

NRL chief executive Andrew Abdo explains the refs' ratings system to Phil Rothfield.

PLAY-THE-BALL SPEED

Rugby league was under fire over wrestling in the rucks. It was a massive problem.

Every club had a specialist martial arts coach to teach tactics to slow down the play-the-ball.

As a result the NRL introduced the six-again rule and the onus was put on referees to improve ruck speed.

That they would be judged on it.

“Everyone wants to watch a free-flowing contest,” Abdo said.

“The game now has a benchmark expectation of 3.5 seconds per play-the-ball.

“Where a referee lets a team get above the benchmark consistently, they will be penalised in their overall score.

“Our data shows there has been a significant improvement and consistency in ruck speed between teams since we introduced this system.

In an effort stop teams from slowing down the ruck, the NRL introduced the six-again rule and called on referees to improve speed. Picture: Getty Images

10-METRE COMPLIANCE

There are around 300 play-the-balls in every game.

The refs are held accountable for keeping a good 10 metres for every one of them.

“We’re aiming for high accuracy here,” Abdo said. “Every time they are below the benchmark they get penalised in their ranking.”

To have a perfect 10m in 300-plus play-the-balls in such a high-speed game is almost impossible. It is understood the benchmark is around 95 per cent accuracy.

KEY DECISIONS

We’re talking here about the howlers. The more costly mistakes that are made in try-scoring or game-changing situations.

The ones that result in a Ricky Stuart or Des Hasler blow-up.

This could be a knock-on, a forward pass, an obstruction or a high tackle.

Even if corrected by the bunker, the referees lose points for getting it wrong.

“We look at the vision of all these incidents,” Abdo said, “We penalise the referees if they are too lenient on say a high tackle. Or if they are too harsh and have overreacted.”

Overall, the referees are getting most calls right. Only 5.5 per cent of tries that are awarded on the field are overturned by the bunker.

NSW skipper Isaah Yeo talks to referee Ashley Klein during State of Origin game two in Perth. Picture: Getty Images

THE BUNKER

The NRL reduced the bunker to four officials after round four of the competition this year.

The number one bunker official Chris Buttler did four games last weekend.

Data shows he has made 322 decisions and judged 316 correctly (98.5 per cent success) at an average decision time of less than 40 seconds.

“We say to them we want them to take the shortest time possible and not make a mistake,” Abdo said. “We’re after speed and accuracy. Get it right. Do it quickly.

“With the timing you want it to be around 30 seconds for tries.”

Reducing the number of officials has had an immediate impact.

“When we started to use fewer people in the bunker the scores started to improve,” Abdo said.

“We saw an instant improvement by decreasing the number of officials (from 95.8 per cent to 98.1 per cent).

“We’re now just selecting our best performers and we have an incredibly high decision accuracy rate.”

Referee Gerard Sutton refers to the bunker to make a decision on a no-try call. Picture: NRL Photos

THE RESULTS

Abdo says standards have steadily improved each year since 2020 with refinements to the ratings.

“The commission originally identified the need to develop a system that would allow us to better analyse the performances of referees,” he said.

“And since then their performances have come a long way. Now the referees understand what areas they are being measured in. It’s a lot clearer.

“We’ve seen a good improvement year on year. We see that in the data and their weekly scores.

“And importantly, the evidence is there to show we now have faster decision making and a higher level of accuracy. And that has given us a better competition.”

COACHING

The data is not just valuable for ranking the referees for appointments.

“The value is unpacking the strengths and weaknesses for each individual ref,” Abdo said.

“Each referee will get a report on their performance from the weekend and it’s used in coaching.

“They’ll know if they have to improve their ruck speed or 10 metres or any other area.

“It’s another reason why we’ve seen a strong improvement.

“These officials overall are doing a tremendous job and they actually thrive on the feedback.”

Originally published as One NRL bunker official achieved a 98.5% success rate - this is how he did it