Virus modelling far from perfect

University of Melbourne Faculty of Medicine, Dentistry and Health Science Dean Professor Shitij Kapur. Picture: Aaron Francis

Last weekend, the Victorian government released its detailed strategy for coming out of lockdown — a roadmap unprecedented in its clarity in terms of case targets and dates.

Like any decision, it has drawn praise from some quarters and criticism from others.

Several discussions have referred­ to a University of Melbourne­ model involved in the decision. We thought it important to clarify what the model says and what it cannot, and also to highlight the ­nuance and inter­pretation­ that must be layered on these models to allow the complex­ choices that leaders must make.

Epidemiological modelling is somewhat like a bank mortgage calculator. You put in the house price; adjust a few parameters (down-payment, interest rate and duration) and out comes a monthly mortgage payment. That calculator can’t tell you whether you should buy the house — it merely informs your judgment. The decision to buy the house depends upon your risk appetite, the many trade-offs and preferences.

Epidemiological modelling is just much more complex. It deals with dozens of parameters and it does not come up with a single ­answer, but provides a likely answer with a range of possibilities.

So it is with the model developed by Jason Thompson, with assistance from professors Tony Blakely and Mark Stevenson, also of the University of Melbourne, and professor Rod McClure from the University of New England, which informed the Victorian government’s decision.

It started with the real number of cases on September 3. It then assumed that 90 per cent of the public would wear a mask; that compliance with social distancing would decrease by about 15 per cent a fortnight; that a third of the cases would be asymptomatic and hence not easily detected; that only 80 per cent of the cases would be detected and told to isolate, of whom only 93 per cent would isolate; and so on and so forth — more than a dozen such assumptions, all carefully chosen based on the best available evidence from around the world.

While this model captured many elements of the real world, it did not capture them all. For example, the model does not distinguish between cases with known contacts versus “mystery cases”, and nor does it distinguish betwee­n cases in aged-care settings from those in the community; or where in Victoria they arise.

These limitations of the model, as well as many others, were openly acknowledged by the authors in the details released.

The value of modelling is that it replaces hunches, intuitions and biases with numbers, parameters and equations. It allows a decision-maker to look at what may happen to those numbers when you lift the restrictions or when you put them back on. It helps build scenarios and outcomes. It guides you as to what to choose, but at the end of the day a decision-maker must make a choice informed by the model and their knowledge of the real world.

Translating model outputs into a decision requires a consideration of many other factors. It requires consideration of acceptable risk. In this case, the risk of overwhelming the public health response, in turn leading to un­acceptable social, economic and health and medical costs. It requires balancing the trade-offs ­between lives affected and livelihoods lost. It entails a choice of how much one can and should constrain the liberties and lives of citizens. These wicked choices cannot be meaningfully reduced to the mathematics of the model. And that is why a model can only inform a decision — not make it.

The Victorian decision has prior­itised suppression of transmission until case numbers are further reduced, with a target of fewer than five new cases over a fortnight by October 26 as a precondition for further loosening of restrictions. It is interesting to examine how it compares to what others have been able to achieve.

No major city in Europe has attempted to suppress transmission to this extent — not London, not Paris or Berlin. Nor have New York or Los Angeles or Chicago; and certainly not Sao Paulo, Mexico­ City or Mumbai. Even the Asian cities lauded for their public health systems and compliant populations — Hong Kong, Singapore, Tokyo and Seoul — have not managed to achieve this strict threshold.

Sydney is different. It never had the case density of Melbourne, and yet it is interesting to note that even Sydney has not reached this threshold. The only major city that has achieved this is Wuhan, a city of 11 million people that endured thousands of cases but imposed perhaps the harshest of lockdowns, barring practically all movement and work.

So, could Melbourne get to fewer than five cases over a fortnigh­t by October 26? Yes, it is possible. But it would be quite a ­remarkable achievement and a model cannot assure that.

Models are only as good as their inputs and assumptions. Many of the two dozen inputs and assumptions that informed this model may change even before the end of the month. Some of these assumptions will improve (such as our testing and tracing ability) and some may worsen (such as social distancing fatigue).

When the new assumptions are entered into the model in late September, it will lead to different predictions. The question is: what should the government do then?

Just as we laud the government for providing clear guidelines based on a September 3 model, we urge the government to keep refining the models, perhaps­ build in considerations such as aged-care versus community, or rural versus metro, into future modelling. And when new information in mid-month is fed into such a model, then perhaps­ a different set of trade-offs may seem more optimal.

After a categorical announcement last Sunday, it is heartening to hear the chief health officers acknowledge that the future dates and numbers are “indicative” and not set in stone.

Because as the noted economist John Keynes is alleged to have said: “When the facts change, I change my mind. What do you do, sir?”

Professor Shitij Kapur is dean and assistant vice-chancellor of health at the University of Melbourne.
James McCaw is a professor of mathematical biology at the University of Melbourne.
The views expressed are their own and not those of their institutions