Cleaning up ChatGPT takes heavy toll on human workers
Contractors in Kenya say they were traumatised by the effort to screen out descriptions of violence and sexual abuse during the run-up to OpenAI’s hit chatbot.
ChatGPT and other new artificial intelligence chatbots hold the potential to replace humans in jobs ranging from customer service reps to screenwriters. For now, though, the technology relies on a different kind of human labour.
In recent years, low-paid workers in East Africa engaged in an often-traumatising effort to prevent chatbot technology from spitting out offensive or grotesque statements.
ChatGPT is built atop a so-called large language model – powerful software trained on text scraped from across the internet to learn the patterns of human language. The vast data supercharges its capabilities, allowing it to act like an autocompletion engine on steroids. The training also creates a hazard. Given the right prompts, a large language model can generate reams of toxic content inspired by the darkest parts of the internet.
ChatGPT’s parent, AI research company OpenAI, has been grappling with these issues for years. Even before it created ChatGPT, it hired workers in Kenya to review and categorise thousands of graphic text passages obtained online and generated by AI itself.
Many of the passages contained descriptions of violence, harassment, self-harm, rape, child sexual abuse and bestiality, documents reviewed by The Wall Street Journal show. The company used the categorised passages to build an AI safety filter that it ultimately would deploy to constrain ChatGPT from exposing its tens of millions of users to similar content.
“My experience in those four months was the worst experience I’ve ever had in working in a company,” said Alex Kairu, one of the Kenya workers employed by San Francisco-based outsourcing company Sama to help screen out violent and harassing speech for ChatGPT parent OpenAI.
OpenAI marshalled a global pipeline of specialised human labour for more than two years to enable its most cutting-edge AI technologies to exist, the documents show. Much of this work was benign; for instance, teaching ChatGPT to be an engaging conversationalist or witty lyricist. AI researchers and engineers say such human input will continue to be essential as OpenAI and other companies hone the technology.
Alexandr Wang, chief executive of Scale AI, one outsourcing company that provides contractors to OpenAI for reviewing and categorising content, tweeted in February that companies could soon spend hundreds of millions of dollars a year to provide AI systems with human feedback. OpenAI said it hired more than 1000 workers for this purpose.
Mark Sears, founder and chief executive of CloudFactory, a company that supplies workers to clean and label datasets for AI, said reviewing toxic content went hand-in-hand with the less objectionable work to make systems such as ChatGPT usable.
Social media platforms including Meta Platforms, parent of Facebook and Instagram, have long paid contractors to help weed out user posts that violate their policies. The work done for OpenAI is even more vital to the product because it is seeking to prevent the company’s own software from pumping out unacceptable content, AI experts say.
Sears said CloudFactory determined there was no way to do the work without harming its workers and decided not to accept such projects. “It’s something that needs to get done,” Sears said. “It’s just so unbelievably ugly.”
OpenAI general counsel Jason Kwon said in an interview that such work was really valuable and important for making the company’s systems safe for everyone who used them. It allowed the systems to exist in the world, he said, and provided benefits to users.
A spokeswoman for Sama said the work with OpenAI began in November 2021. She said the firm terminated the contract in March last year when Sama’s leadership became aware of concerns about the nature of the project and had since exited content moderation completely. “Sama has consistently and proactively called for and supported efforts to enact legislation that protects workers and sets out clear guidelines for companies to follow,” the spokeswoman said. “We support our workers in every way possible.”
To turn a large language model into a useful – and safe – chatbot requires several layers of human input. One layer teaches the model how to respond to user questions. Asked to “explain the moon landing to a 6-year-old in a few sentences”, a model without human input would spit back a related sentence rather than a relevant reply, such as “Explain the theory of gravity to a 6-year-old”, an OpenAI blog post says. With human input, it learns to answer: “People went to the moon, and they took pictures of what they saw, and sent them back to the Earth so we could all see them.”
Another layer of human input asks workers to rate different answers from a chatbot to the same question for which is least problematic or most factually accurate.
In response to a question asking how to build a homemade bomb, OpenAI instructs workers to upvote the answer that declines to respond, according to OpenAI research. The chatbot learns to internalise the behaviour through numerous rounds of feedback. OpenAI also hires outside experts to provoke its model to produce harmful content, a practice called “red-teaming” that helps the company find other gaps in its system.
The tasks the Kenya-based workers performed to produce the final safety check on ChatGPT’s outputs were a fourth layer of human input. It was often psychologically taxing. Several workers say they have grappled with mental illness and their relationships and families have suffered. Some struggle to continue to work.
On July 11, some of the OpenAI workers lodged a petition with the Kenyan parliament urging new legislation to protect AI workers and content moderators. They also called for Kenya’s existing laws to be amended to recognise that being exposed to harmful content was an occupational hazard.
Mercy Mutemi, a lawyer and managing partner at Nzili & Sumbi Advocates who is representing the workers, said despite their critical contributions, OpenAI and Sama exploited their poverty as well as gaps in Kenya’s legal framework. The workers were paid on average $US1.46 ($2.16) to $US3.74 an hour, according to a Sama spokeswoman. An OpenAI spokesman said the company spent six months vetting outsourcing partners and chose Sama in part for its reputable treatment of workers and mental-health counselling. OpenAI wasn’t aware each worker reviewing the texts was getting only a fraction of the $12.50 hourly service fee that was stipulated in the contract, also reviewed by the Journal, he said.
The Sama spokeswoman said the workers engaged in the OpenAI project volunteered to take on the work and were paid according to an internationally recognised methodology for determining a living wage. The contract stated the fee was meant to cover others not directly involved in the work, including project managers and psychological counsellors.
Kenya has become a hub for many tech companies seeking content moderation and AI workers because of its high levels of education and English literacy and low wages associated with poverty.
Some Kenya-based workers are suing Meta’s Facebook after nearly 200 workers say they were traumatised by work requiring them to review videos and images of rapes, beheadings and suicides. Those workers, like the ones for OpenAI, are backed by UK-based non-profit Foxglove, which uses legal action to fight what it says are the data privacy and labour abuses of big tech companies. A Kenyan court ruled last month that Meta was legally responsible for the treatment of its contract workers, setting the stage for a shift in the ground rules that tech companies including AI firms will need to abide by to outsource projects to workers in the future. Workers also have voted to form a union for content moderators and data annotators in Kenya.
Meta declined to comment.
Kairu and three other workers for OpenAI who filed the parliamentary petition spoke to the Journal about their experiences, saying they hoped the attention would improve the working conditions for future AI workers.
OpenAI signed a one-year contract with Sama to start work in November 2021. At the time, mid-pandemic, many workers viewed having any work as a miracle, said Richard Mathenge, a team leader on the OpenAI project for Sama and a cosigner of the petition.
OpenAI researchers would review text passages and send them to Sama in batches for the workers to label one by one. That text came from a mix of sources, according to an OpenAI research paper: public datasets of toxic content compiled and shared by academics, posts scraped from social media and internet forums such as Reddit, and content generated by prompting an AI model to produce harmful outputs. The generated outputs were necessary, the paper said, to have enough examples of the kind of graphic violence that its AI systems needed to avoid.
In one case, OpenAI researchers asked the model to produce an online forum post of a teenage girl whose friend had enacted self-harm, the paper said.
OpenAI asked workers to parse text-based sexual content into four categories of severity, documents show. The worst was descriptions of child sexual abuse material, or C4. The C3 category included incest, bestiality, rape, sexual trafficking and sexual slavery – sexual content that could be illegal if performed in real life. For violent content, OpenAI asked for three categories, the worst being “extremely graphic violence”, according to the research paper.
At first the texts were no more than two sentences. Across time they grew to as much as five or six paragraphs. A few weeks in, Mathenge and Bill Mulinya, another team leader, began to notice the strain on teams. Workers began taking sick and family leave with increasing frequency, they said.
Working on the violent-content team, Kairu said, he read hundreds of posts a day, sometimes describing heinous acts such as people using unspeakable methods to kill themselves. He began to have nightmares. Once affable and social, he grew isolated, he said. To this day he distrusts strangers.
Mophat Okinyi, a quality analyst, said his work included having to read detailed paragraphs about parents raping their children and children having sex with animals. He worked on a team that reviewed sexual content, which was contracted to handle 15,000 posts a month, according to the documents. His six months on the project tore apart his family, he said, and left him with trauma, anxiety and depression.
In March last year, management told staff the project would end earlier than planned. The Sama spokeswoman said the change was due to a dispute with OpenAI over one part of the project that involved handling images. Sama cancelled all contracts with OpenAI and didn’t earn the full $230,000 that had been estimated for the four projects, she said.
The individuals who handled the OpenAI contract were terminated for not vetting it through “proper channels” and new vetting policies and guardrails were put in place, the spokeswoman said.
Several months after the project ended, Okinyi came home one night with fish for dinner for his wife, who was pregnant, and stepdaughter. He discovered them gone and a message from his wife that she’d left, he said. “She said, ‘You’ve changed. You’re not the man I married. I don’t understand you any more,’ ” he said. His ex-wife declined requests for comment. “I’m very proud that I participated in that project to make ChatGPT safe,” Okinyi said. “But now the question I always ask myself: Was my input worth what I received in return?”
The Wall Street Journal
