The cybersecurity gap that threatens to derail Australia’s multibillion-dollar AI boom

Corporate Australia faces an ‘AI paralysis’ crisis as companies struggle to move beyond lab testing, which is threatening $600bn in forecast economic benefits by 2030.

AI Image of a cyber attack. Picture: Google Gemini

Australian companies are experiencing a crisis of conviction over artificial intelligence, despite the technology’s much-hyped potential to turbocharge productivity and inject hundreds of billions of dollars into the economy by the end of the decade.

Despite billions of dollars in investment globally and executive mandates to “do something with AI,” a vast number of projects are failing to make the leap from the internal lab to secure, production-ready systems. It’s a problem Microsoft calls “AI paralysis”. 

Telstra Vicki Brady expects every employee at the telco to use AI. Picture: Martin Ollman.

Complicating matters is the risk of things going wrong when AI enters production. Apple TV+’s Morning Wars highlighted this with dramatic effect, when the technology cost the CEO of a fictional media company depicted in the series her job after AI went rogue.

In reality, the broader rollout of the technology is proving just as prickly. Commonwealth Bank’s AI experiment delivered a masterclass in corporate self-sabotage, when the technology displaced 45 jobs.

CBA discovered, in a very public and humiliating wat, that the headlong rush into AI adoption without a deeply considered human element could quickly transform perceived benefits into a public-relations nightmare, while alienating customers and staff.

Others like Telstra are experiencing more success. The telco has bought more than 20,000 licences of Microsoft’s Copilot AI assistant for its staff in an effort to boost output. CEO Vicki Brady said she regularly uses to the technology to prepare for earnings calls and expects every employee to do the same as part of their jobs.

Airwallex founder and chief executive Jack Zhang.

Airwallex boss Jack Zhang has also told staff to use AI or risk redundancy as he prepares to unleash an AI-powered chief financial officer for businesses. Airwallex has also commission a YouGov survey of Australian business, which found that 19 per cent of companies that responded expected to appoint a chief AI officer to work alongside the CFO in the near future as many sought to launch more AI in their operations.

But many companies are not getting to that point, which threatens to derail the federal government’s forecast of AI injecting up to $600bn a year into the economy by 2030.

Ameya Talwalkar, chief executive and co-founder Cequence Security – a company backed by Titanium Ventures, formerly Telstra’s venture capital arm – said the rush to build an AI prototype, often driven by a genuine “fear of missing out”, was proving far easier than the deployment phase.

He said a lot of internal AI projects designed for customer success, sales efficiency, or engineering productivity, showed early promise but inevitably stalled when attempts were made to scale it up.

“Enterprises are realising that building an AI prototype can be relatively straightforward. However, more and more are discovering that deploying it into production, whether for internal use or customer-facing applications to truly capture value, is an entirely different challenge,” Mr Talwalkar said.

Cequence chief executive and co-founder Ameya Talwalkar.

He said the challenge lay in the technical debt and immense security risk created when organisations neglected critical infrastructure and governance.

This sentiment is echoed by an independent poll of Australian technology and security executives conducted by Okta. The Okta AI Security Poll revealed that the organisational framework for managing this risk was almost non existent. A total of 41 per cent of respondents said no single person or function was currently responsible for AI security risk in their organisation. Without clear accountability, projects flounder.

The existential risk of this governance vacuum was highlighted by the chief executive of Okta, Todd McKinnon, who highlighted the cautionary tale of McDonald’s. The fast-food behemoth’s AI agent “Olivia” exposed millions of job applicants’ data to hackers due to a shocking vulnerability: a “123456” password and a complete absence of multi-factor authentication.

The Okta data underscores how widespread this security blind spot is in Australia. The poll found that only 18 per cent of organisations were confident they could detect whether an AI agent acted outside its intended scope, and just 10 per cent said their identity systems were fully equipped to secure non-human identities like AI agents and bots.

Okta chief executive Todd McKinnon.

Furthermore, “shadow AI” – the use of unapproved or unmonitored tools – was identified as the top security blind spot for 35 per cent of local executives.

The urgency of the situation was driving companies like Cequence to emphasise that API and agent security must be integrated from the beginning, not rushed in as an afterthought.

“In production you want proper authentication authorisation, monitoring, logging and security to prevent malicious/misbehaving agents from causing critical issues,” Mr Talwalkar said.

“We have found many organisations start with internal projects in an attempt to better manage risk but, while agentic AI can truly bring relief to key pain points, critical enterprise implementation issues must be handled before the tech can deliver on its promise.”

Originally published as The cybersecurity gap that threatens to derail Australia’s multibillion-dollar AI boom