Chatting up a wave of fear, but ChatGPT not passing all tests

ChatGPT is a conversational artificial intelligence software application developed by OpenAI. Picture: AFP

It’s the chatbot that helped Jeremy Hunt, the UK Chancellor of the Exchequer, write his recent speech on the UK economy and is helping millions of others with homework, computer code, essays, poems and business presentations.

It has passed MBA, Bar and US medical licensing exams, has been banned from universities, and has spawned versions that can give you bespoke recipes, build apps and even co-host a podcast.

Two months after the release of ChatGPT, the artificial intelligence program is still on the lips of the tech world, supercharging its hype.

ChatGPT is what is called a large language model. It has been trained on billions of words from the internet and then refined by humans. Its power comes from being able to write sentences because it can accurately predict the next word to write, like auto-complete but on a huge scale. Users are able to ask it a wide range of questions and it returns an answer almost instantly.

OpenAI, the US company that unleashed it, is looking to create a premium version that may cost $US42 ($59) a month. But beyond the fever pitch of excitement, some are urging caution.

The award for chatbot party-pooper goes to Yann LeCun, Meta’s chief AI scientist. His reaction to ChatGPT? “A flashy demo,” he told the Big Technology Podcast. “From the scientific point of view, GPT is not a particularly interesting scientific advance.”

His argument is that the chatbot is just regurgitating text it’s been trained on and has no understanding of the real world, so lacks basic intelligence. Most of human intelligence is not text-related, he says, and involves a major control planning system created by evolution. None of that is captured by any current AI system.

LeCun contends that Open AI released the chatbot to please its paymasters, principally Microsoft. He adds that most of the technologies it uses were invented at Google, Meta and Deepmind, an AI company now owned by Google. If this sounds like the sour grapes, he does concede ChatGPT “is very well engineered” but says Google and Meta, which possess similar models, are too cautious to release them. “You might … ‘why aren’t there similar systems from, say, Google and Meta?’,” he told a US conference last week. “And the answer is, Google and Meta both have a lot to lose by putting out systems that make stuff up.”

He was referring to the chatbot’s habit of “hallucinating” – giving wrong answers with confidence. LeCun knows about this as Meta recently released Galactica, a conversational AI engine for academia, but pulled it after users generated papers on the benefits of suicide, anti-Semitism and eating crushed glass.

If Meta is relaxed, Google appears to be in a mini-panic. Its founders, Larry Page and Sergey Brin, who left their daily roles in the company in 2019, are back trying to devise an AI strategy that tackles the ChatGPT threat. Many believe the underlying technology represents the future of search, rather than the ad-driven Google model.

Sundar Pichai, Google’s chief executive, has declared a “code red”, akin to pulling the company fire alarm, according to The New York Times. The emergency looks real, too, with Microsoft’s announcement it will incorporate ChatGPT into search engine Bing.

While the AI giants slug it out, ChatGPT has been studying hard. A professor from the Wharton School of Business in Pennsylvania put it to work on its MBA (Master of Business Administration) final exam. It did an “amazing job” answering basic business questions on case studies but less well on maths and advanced analysis, according to the study’s author, Christian Terwiesch. Final grade? B or B minus, he says.

It also makes for a passable lawyer, having earned a C+ from the University of Minnesota on its Bar exam, although that would have seen a real student placed on academic probation. It also passed the US medical licensing exam.

Some academics have taken to giving ChatGPT co-author credits, creating a headache for journals. One institution that has drawn a line in the sand, is, ironically, the International Conference on Machine Learning. It has banned authors from using AI tools such as ChatGPT to write scientific papers.

Sam Altman, one of the co-founders of OpenAI, conceded recently that he and many observers on the future of work were wrong: the robots didn’t come for the physical jobs first, they are coming for those who sit in front of a computer. Technology news site CNET ran into trouble this month after using a text generator like ChatGPT to write financial articles under the CNET Money staff byline. Unlike the Associated Press, which started using AI in 2014 to write stories about corporate earnings, CNET didn’t clearly tell its readers, or most of the newsroom, and the articles were filled with errors. It has paused the project, which ran into many of the dilemmas surrounding the technology: Can it be trusted? What are its sources? What jobs will it replace?

Gary Marcus, professor of psychology and neural science at New York University, has been sounding a warning. He told fellow professor Scott Galloway’s podcast, the Prof G Pod: “People are over-attributing to ChatGPT an intelligence that’s not really there.” More bluntly, he contends that “it does make up so much bullshit”.

“I don’t think it can solve the truth problem,” he said. “It’s not actually analysing its dataset, saying, ‘this thing that I’m saying, is it consistent with what I know?’. And this lack of a validation step in my mind is fatal for making it a serious, full-service search engine.”

For those at the coalface of AI, this is still a big moment. Nathan Benaich, co-author of the State of AI Report and founder of venture firm Air Street Capital, said: “I think it’s very significant for a lot of software companies that I work with, particularly those that are ‘software as a service’ businesses that weren’t necessarily born with machine-learning in their product.”

ChatGPT also co-presents a podcast with Reid Hoffman, the LinkedIn founder and Open AI investor. This is its spin when asked about its impact on jobs. “Instead of replacing human workers, AI is being used to assist and enhance their abilities. By taking on repetitive and mundane tasks, AI allows humans to focus on more complex and creative endeavours … AI can also generate new ideas and insights that humans may not have been able to come up with on their own.”

The Times