The problem steering robo-debt is there are no humans at the wheel

Robo-debt is headed for the courts. A class action will argue the federal government unjustly enriched itself by taking up to $300m from thousands of Centrelink clients. The case has the potential to turn a major money-spinner for the commonwealth into a much costlier exercise.

Governments have long struggled with ensuring Australians receive the correct social security benefits and that overpayments are returned. With millions of welfare recipients, it is impossible for agencies to be fully aware of the circumstances of each person. Reliance has been placed instead on personal disclosure and manual review of people at the highest risk of being overpaid. The system was time-consuming, expensive and often ineffective.

The government’s answer to this problem since 2016 has been robo-debt. It is an automated system based on an algorithm that crossmatches data on annual income held by the Australian Taxation Office with income reported to Centrelink. Where the tax income is higher, it is taken to mean that the people receiving benefits had been overpaid and so owe a debt to the government.

Systems such as this have become increasingly common around the world. The main driver is the financial gains they provide to government. Australians have been issued with more than a half-million robo-debt notices, enabling the government to collect $2bn in additional revenue, especially welcome when the Coali­tion has been determined to turn the budget into surplus.

Automated systems also can provide other advantages. Humans are prone to error and bias, whereas automated systems promise to manage government entitlements in a way that is more accurate and fairer. Automated systems can apply rules strictly so people across the country in the same circumstances are treated alike. They are less effective, though, in applying discretion in special cases or where there is a need for compassion. This means human judgment and empathy are still required.

The problem lies not in the idea of automation but in how robo-debt has been implemented. A windfall for the government has produced high levels of distress in the community, particularly among elderly, poor and vulnerable people.

One problem is that the system was based on an erroneous formula that averaged a person’s earnings across a series of fortnights, rather than using what the person actually earned during a particular fortnight. This produced discrepancies that meant many people wrongly received a letter of demand from the government seeking repayment. The problem has been exacerbated by the fact the letters were intimidating and poorly worded.

The result is an automated system involving hundreds of thousands of Australians with an unacceptably high error rate. As the government has acknowledged, 20 per cent of its robo-debt recovery letters were sent to people who did not owe any money.

This was compounded by the absence of sound procedures for dealing with these errors. There was no human checking of the decision to issue a debt notice, and the online portal for people to challenge the government claim was hard to use. The onus also was put on people to show why they did not owe the debt. Many were incapable of doing this or did not do so because they were frightened by the prospect of being pursued by debt collectors or the government.

One lingering question over robo-debt is whether it is lawful. There is a credible legal argument that the government is acting outside the law by demanding the repayment of money it is not actually owed. It also may be that the government is in the wrong by asserting a debt and then putting the onus on people to show why that is not correct. These issues no doubt will be tested as part of the class action.

The fact these arguments can be mounted shows the government failed to properly lay the legal foundations for the scheme. Government Services Minister Stuart Robert says the system is working as it should. He is right that the government has a responsibility to collect its debts, but it is hard to see how this extends to demanding payment from people who do not owe a cent. Special laws and better procedures were needed from the beginning.

These concerns do not undermine the overall case for data matching and automation. They instead suggest that robo-debt should have been better designed.

The government could have looked to other countries with a stronger record. Sweden, for example, uses an automated system to manage student financial aid. Students are a good group for these systems because they are generally familiar with technology. However, even then, human officers are involved in the process to guide people through the e-service. This ensures decisions are based on clear, public rules and that a human confirms and takes responsibility for each decision.

Robo-debt compares poorly with such examples. It bears the human failings of its designers. Most important, we need to remember that automated systems, especially those dealing with vulnerable and disadvantaged people, still require high levels of human involvement. Saving money must not come at their expense.

George Williams is dean of law at the University of NSW.