Royal Australasian College of Physicians exam scores epic fail

A computer failure during a crucial test affected hundreds of trainee physicians.

The RACP exam failure came after debate about pressure medical students face.

About 1200 trainee physicians in Australia and New Zealand have had to reschedule crucial exams after a serious failure with the computer-based testing process this week, throwing their lives and hospital schedules into chaos.

The Royal Australasian College of Physicians had engaged the company Pearson Vue to conduct its divisional written examination across multiple sites via computers on Monday.

But several hours into the exam a problem was detected at some sites and, although many trainees were urged to continue, the exam ultimately was cancelled.

The computer failure not only had implications for trainees — concerns for their mental health have been paramount this week — but is likely to have ongoing ramifications for the RACP and Pearson Vue. Similar incidents in the past have ended up in the courts.

The divisional written examination tests a trainee’s knowledge in adult internal medicine or pediatrics and child health at the end of basic training.

This assessment, completed before trainees undertake the divisional clinical examination, is the gateway for progression to advanced training.

Not only does the exam cost each trainee about $1800, it requires months of study and in some cases logistical work for the junior doctors to be able to set aside the six-plus hours needed just to sit the test.

Every trainee is photographed, required to provide proof of identification and their signature, and put under strict supervision during the exam.

The computer failure also comes after considerable debate in the professions about the pressure medical students, trainees and junior doctors are under, and the impact on their psychological wellbeing. The RACP previously has acknowledged the potential for trainees to have increased anxiety undertaking a computer-based exam for the first time.

The Weekend Australian spoke to several trainees affected by the failure and heard varying accounts of what happened.

At some sites, invigilators kept trainees informed and announced the exams were being cancelled, while others received hastily printed leaflets confirming the cancellation. The shock and disappointment on display in the venues led to several days of outrage on social media.

Trainees had taken leave from hospital jobs to prepare for the exam and many planned to take a break from their studies before hopefully embarking on the clinical component. For some, that would have meant travelling overseas, or spending time with family and friends, and the implications of having to reschedule the exams were not immediately apparent.

The RACP ordered Pearson Vue to report back on the computer failure within 48 hours and promised its own investigation. It initially rescheduled the exams for March 2.

RACP president Catherine Yelland says the college is listening to trainees and aware of the emotional and financial impact the computer failure has had on them.

“I’ve done that exam,” Yelland tells The Weekend Australian. “I know how important it is and how tense everyone is around it.”

Australian Medical Association Council of Doctors in Training chairman John Zorbas questioned why there was no the capacity to revert to paper-based exams in the event of computer failure.

“This isn’t an exam at the end of the week, this is an exam people will spend one, two, sometimes three years studying for,” Zorbas says, emphasising that trainees have to prepare psychologically for a conclusion.

“For it to all fall apart midway through because of a technical glitch is unfathomable.”

While paper-based exams were available, Yelland suggests they were not available as a contingency measure because the venues specifically were set up for computers.

The rescheduled exams will revert to paper-based exams.

“I don’t think that there would be any acceptance that we would, so soon after the event, and before we have all the answers, attempt a second computer-based examination,” Yelland says.

In a webinar ahead of the exam, RACP officials suggested there were risks associated with paper and computer-based exams, but the move to computer-based exams promised improved accuracy, efficiency and security.

“There are a number of different contingency plans that we have in place in case of computer failure or power outage … and if there is any issue, rest assured that will be handled swiftly,” one official responded to a question during the webinar.

Yelland says the RACP had followed other colleges in moving exams online and outsourced delivery of that exam to Pearson Vue. “The central issue was a computer problem between the first part of the examination and the second,” she says.

A spokeswoman for Pearson Vue did not return calls by deadline. In the webinar, the RACP officials said Pearson Vue was selected because of its extensive experience delivering computer-based exams around the world, including for major health organis­ations. They promised to survey trainees on their experiences.