Cleverest chatbot flunks French philosophy exam

While teachers are panicking over cheating with computer-generated answers, the latest robot is no match for the Baccalaureat exam.

The robot’s essay had “all the faults that we try to fight when we explain to students what a dissertation is”, said Paris University lecturer, Olivier Dhilly. Photo: iStock

Teachers may be alarmed over cheating with computer-generated answers but the latest robot is no match for the Baccalaureat philosophy examination.

ChatGPT, released in late November by OpenAI in San Francisco, was instructed to produce a dissertation in answer to a question from last year’s “Bac Philo”, the rigorous test that is compulsory for most candidates for the French equivalent of A-levels. Students had four hours to discuss “Is it up to the state to decide what is fair?”

Le Figaro handed the robot’s essay to Olivier Dhilly, a senior sixth form teacher and lecturer at Paris University, who said that it lacked the reasoned argument that is expected of sixth formers. Dhilly said: “What it [ChatGPT] came up with was . . . based on really questionable points.”

pic.twitter.com/m7tquTE3Xf

— ChatGPT ✨ (@ChatGPTBot) December 15, 2022

The chatbot, which answers in prose in several languages, cited the precepts of Thomas Hobbes, Plato and Arisotle to lay out an apparent argument. “They were general . . . and they led to a mini-recitation of theories that were not connected to the precise question,” Dhilly said. The robot’s essay had “all the faults that we try to fight when we explain to students what a dissertation is”.

Teachers in the United States and Europe are in near panic over AIgenerated answers to logic and maths problems. State schools in New York City have blocked ChatGPT access because of the “impact on student learning and concerns regarding the safety and accuracy of content”.

The Times