‘Impossible’: New AI detection tool slammed by experts
A new AI paraphrasing detection tool designed to help educators identify plagiarism has been slammed by experts who claim it is “impossible to detect artificial intelligence”.
Education
Don't miss out on the headlines from Education. Followed categories will be added to My News.
A new AI paraphrasing detection tool designed to help educators identify plagiarism has been slammed by experts who claim it is “impossible to detect artificial intelligence”.
It comes as thousands of university students across Queensland under investigation for using AI said the process to determine the integrity of their assessments was delaying their studies and graduations with some saying they were falsely accused.
The new tool was created as a way for Turnitin – a web-based plagiarism prevention system used in educational facilities – to catch out students and staff who plagiarise using AI programs such as Chat GPT.
But Charles Darwin University Artificial Intelligence Expert Stefan Popenici said AI detection tools had been “constantly contradicted by evidence and scientific studies”.
“The whole thing with generative AI is that Chat GPT is a product of Open Ai who have explicitly said AI detectors do not work.”
“Since then (when Open AI said the detectors don’t work), almost a year ago, educators use AI detectors and it puzzles me why they still do that,” he said.
Mr Popenici said Turnitin had, for a long time, made false promises regarding plagiarism detection for artificial intelligence.
“It’s closely impossible to detect AI and there’s an unacceptable high score of false positives,” he said.
“If you look at the amount of money these companies spend, its easy to understand why they come out with these things to stay relevant that are constantly, constantly contradicted by evidence and scientific studies.”
Mr Popenici expressed his empathy to students who had been falsely accused of using AI causing “distress”
Meanwhile, Asia-Pacific Turnitin Regional Vice President James Thorley urged educational institutions to keep in mind that Turnitin should not solely be relied on to catch AI plagiarism.
“It (AI detection) is because of the predictability,” he said.
“ … a human would rewrite in a much more unpredictable way than the AI will.”
Mr Thorley said the information that they’re providing is just one data point.
“It’s very important that you are looking for other pieces of evidence if you’re if you suspect that there may be some academic misconduct,” he said.