Coronavirus Australia: call for ‘failed’ doomsday health forecast models to be abandoned

Top statistician says modelling that contributed to the sharpest economic downturn in a century must immediately be abandoned.

Shoppers walk past retail signage in Sydney. Picture: AAP

A top statistician has blasted “doomsday” COVID-19 forecasts that pressured governments to lock down their economies for months as a “failure”, and said ­predictions hospitals would be overwhelmed with patients were “entirely misinforming”.

Sally Cripps, a professor of statistics at the University of Sydney, has weighed into the debate over coronavirus modelling that has contributed to the sharpest economic downturn in a century, in ­research written with Stanford ­epidemiologist John Ioannidis.

“Despite involving many excellent modellers, best intentions and highly sophisticated tools, forecasting efforts have largely failed,” they wrote in a draft analysis published earlier this month.

“Implausible, exaggerated fore­casts should be abandoned — otherwise they may cause more harm than the virus itself.”

‘Despite involving many excellent modellers, best intentions and highly sophisticated tools, forecasting efforts have largely failed’: Sally Cripps, a professor of statistics at the University of Sydney. Source: Twitter

The unemployment rate jumped to 7.1 per cent in May, a 19-year high, as lockdowns imposed by state governments in March to avoid predictions of tens of thousands of deaths from COVID-19 hammered the labour market. So far, 102 people have died from, or with, COVID-19 in Australia.

Even with strict, mandatory ­social-distancing measures, the Doherty Institute modelling that informed the national cabinet’s decisions suggested about 5000 intensive care beds would be required, when fewer than 100 have been used.

“Most hospitals maintained largely empty wards, waiting for tsunamis that never came,” the ­authors wrote, pointing to hugely pessimistic forecasts for New York, one of the cities worst hit by the coronavirus.

In late March, Columbia University projected 136,000 beds would be required; the maximum used was less than 12,000, and at the peak of the virus’s impact a sixth of hospital beds were free.

This is terrific discussion on forecasting@nntaleb Single pt forecast for fat tailshttps://t.co/aNWDbGPJ1i
and John Ioannidis, Sally Cripps, Martin Tanner
Forecasting for COVID19 has failedhttps://t.co/mblQhQszGX

— Vinay Prasad (@VPrasadMDMPH) June 14, 2020

Massachusetts General Hospital predicted 23,000 deaths within months of the US state of Georgia reopening, yet the actual number was 896, the authors said.

They argued the most widely reported models “lacked transparency”, used poor quality data and focused on deaths rather than the concept of “quality adjusted life years”, which factors in the age of those affected.

“The Spanish flu (with a death toll more than 50 million globally) killed young people with an ­average age of 28, and its burden in terms of number of quality ­adjusted person years lost was about 1000-fold higher than COVID-19,” they said.

“Isolating the infectious impact from all other health, economy and social impacts is dangerously narrow-minded,” they argued, pointing to the likelihood of “starvation and lack of control for other infectious diseases like tuber­culosis and malaria”.

Modelling for swine flu predicted 3000 to 65,000 deaths in the UK, yet 457 died.

“It is surprising that epidemic forecasting has retained much credibility among decision-­makers, given its dubious track ­record,” the authors wrote.