AI Medical Scribe that’s used by doctors can make ‘hallucinations’ or errors

Australian doctors are being warned about using artificial intelligence software to record clinical notes during patient consultations after dangerous errors have been reported.

The Royal Australian College of General Practitioners warns that the tool must be used with caution.

Doctors are being warned about using AI software to record clinical notes during patient consultations after dangerous errors have been reported in the transcriptions.

When making appointments via popular online booking platforms, patients across the state are being asked if they consent to AI Medical Scribe Software being used during the consultation with the doctor.

The scribe is a tool that can convert a conversation with a patient into the clinical notes and is designed to lessen the administrative load for the doctor and allow them to focus on the patient.

The Royal Australian College of General Practitioners warns that while the tool can help ease GP burnout it must be used with caution.

The college advises that all GPs review the notes for accuracy.

“The RACGP has released guidance on the use of AI scribes in general practice, which advises patients must be informed and provide consent for AI scribes to be used during a consultation and GPs should always review the notes produced. These tools must also be compliant with Australian laws for safe data collection and storage,” RACGP president Michael Wright said.

Australian Doctor has published AI scribe “hallucinations” or errors as reported by doctors.

One AI report made up a neurological exam that never happened. Another transcript recorded the wrong side of a patient’s breast cancer.

Doctors consent form for AI.

The Australian Health Practitioner Regulation Agency (AHPRA) and National Boards support the safe use of AI in healthcare.

The Australian Medical Association’s position statement on artificial intelligence in healthcare

outlines that the ultimate responsibility for any misdiagnosis or mistreatment must be clear — the ultimate decision on patient care should always be made by a human.

Former RACGP president Nicole Higgins said that the administrative burden on GPs needs to be reduced urgently.

“Our annual Health of the Nation report found GPs are increasingly reporting the administrative workload and associated stress among their greatest concerns,” she said.

“However, AI scribes need to be used with caution as it is an emerging technology and there are risks.

“It’s important to understand AI scribes cannot replace the work GPs do to prepare clinical documentation, and doctors must carefully check their output for accuracy, as they can make mistakes.”

A Brisbane patient, who did not wish to be named, said he wasn’t ready to put full faith in artificial intelligence keeping accurate notes for his health record.

“I’m glad there is an option to opt out of it when making an appointment,” he said.