ASIC warning on AI embrace as regulator runs ruler over new tech

Financial services companies are exposing themselves to action from the corporate regulator after it found many had introduced AI technologies with little thought to the consequences or controls.

ASIC chairman Joe Longo. Picture: NewsWire / Josie Hayden

Financial services companies are exposing themselves to court action from the corporate regulator after its sweeping review found many had introduced AI technologies with little to no thought to the consequences or controls.

In a report published on Tuesday, the Australian Securities & Investments Commission says nearly half the companies surveyed did not have policies in place governing fairness or bias in their AI technologies, with even fewer disclosing the use of the programs to consumers.

ASIC chairman Joe Longo says the uptake of AI is accelerating but it is not being matched with an increase in policies or critical thinking governing its use.

“What ASIC really wants to avoid is licensees using AI in a way that amplifies existing risks to consumers and introduces new ones,” he said. “We don’t want to see consumer harm caused by AI models trained on unconventional data sources such as social media making inappropriate credit or insurance decisions.”

Mr Longo said it was “clear that work needs to be done” in a bid to ensure AI technologies in use across the financial system were governed correctly.

“When it comes to balancing innovation with the responsible, safe and ethical use of AI, there is the potential for a governance gap – one that risks widening if AI adoption outpaces governance in response to competitive pressures,” Mr Longo said.

“Without appropriate governance, we risk seeing misinformation, unintended discrimin­ation or bias, manipulation of consumer sentiment and data ­security and privacy failures, all of which has the potential to cause consumer harm and damage to market confidence.”

Already insurers, banks and other financial service providers are using AI-powered technologies for decision-making and claims management services.

ASX-listed insurer QBE rolled out generative AI platforms across its brands in December last year, reviewing broker submissions for underwriting claims.

QBE has rolled out generative AI across the insurer.

The ASIC AI report finds the extent to which AI is being used across the financial system “varied significantly”, with researchers revealing there was a shift towards “opaque techniques” in its implementation.

ASIC finds overall AI is being used in a “quite cautious” fashion, but the regulator reveals “some gaps in how licensees assessed risks to consumers from AI”, warning that a risk of consumer harm was growing.

The regulator singled out the calculation of borrower risks by some lenders, saying the preparation of credit scores by AI-powered systems “had the potential to result in consumers being refused credit or offered less credit than they otherwise would have been”.

ASIC found one company surveyed had only investigated its AI model 10 months after deployment, noting the company actually had “limited understanding” of how third-party platforms were being used, while the lender had incomplete documentation and lacked governance and monitoring of the platform. ASIC said only 12 licensees had policy documents or checklists for their use of AI.

“In some cases, references were principles-based, and it was not clear how consideration of these principles was embedded into operations,” the report said.

Only 10 licensees had documented requirements to disclose the use is AI, with ASIC noting “some only prompted consideration of whether disclosure is appropriate and did not prescribe an approach to disclosing”.

The report also found no licensees had implemented systems specifically to contest AI decisions. “Some licensees identified the risk of an incorrect model output, but noted the consumer could contest it, or staff could override it” the report said.

“However, they did not consider the potential harm if the model output caused a consumer to abandon their transaction altogether, potentially without knowing they could contest it (or indeed that AI was used).”

ASIC found seven companies had set up or were in the process of setting up internal committees to oversee their use of AI within the business.

But the review found several showed poor practices, meeting infrequently or with a poorly defined mandate.

ASIC’s review said the regulator would take a “technology neutral” approach, pointing to current rules for the financial system.

“There are several existing regulatory obligations that are relevant to licensees’ safe and responsible use of AI – in particular, the general licensee obligations, consumer protection provisions and directors’ duties,” the report said.

Originally published as ASIC warning on AI embrace as regulator runs ruler over new tech