For whom the bot trolls – a social media challenge
Our lives are controlled by social media algorithms that prioritise harmful content and can be easily manipulated. But is big tech finally going to be held accountable?
In December 2021, a Philadelphia mother found her 10-year-old daughter’s lifeless body on the floor of her bedroom.
The girl had been fixated on her iPhone, scrolling for hours, and had attempted to complete the “blackout” TikTok challenge in which young users choked themselves with belts, towels or other objects until they passed out.
She was rushed to hospital, admitted and died five days later.
Addressing reporters at the time, her mother, Tawainna Anderson, said she could not stop replaying the moment she found her daughter’s body. She announced her plans to sue TikTok’s Chinese parent company, ByteDance, to hold it accountable for the death.
“It is time that these dangerous challenges come to an end so that other families don’t experience the heartbreak we live (with) every day,” she said.
Social media algorithms have gripped our attention and taken power of how we act, talk and think. They have compelled users – particularly children – to participate in senseless acts for likes, shares, comments and kudos from their friends. And while platforms have long argued they should not be held responsible for the actions of their users outside the app, that may be about to change.
Anderson – and, depending on which way you look at it, the wider community – got a win last week when 3rd US Circuit Court of Appeals judge Paul Matey ruled TikTok could be held responsible for the content it fed its users.
The judge said while US federal law “provides TikTok immunity from suit for hosting videos created and uploaded by third parties”, it did not protect the platform from being held to account for its alleged “knowing distribution and targeted recommendation of videos it knew could be harmful”.
The blackout challenge reportedly killed 20 children between 2021 and 2022, but it is hardly the first dangerous trend to go gangbusters on the platform.
A 13-year-old girl was left nearly paralysed after completing the “skullbreaker” challenge in 2020, after she jumped in the air and two of her friends kicked her legs out from under her. Another child was hospitalised with broken bones after completing the same task.
Teenagers have been participating in the act of “beezin’ ”, where they rub Burt’s Bees lip balm on their eyelids before they go out to enhance the effects of alcohol. Following numerous cases of eye inflammation, irritation, milia and even vision loss, Burt’s Bees released a statement: “There are lots of natural things that probably shouldn’t go in eyes – dirt, twigs, leaves, food – and our lip balm.”
Experts say platforms allow toxic messaging from online personalities to thrive, with algorithms often favouring outrageous content as it gets clicks.
Macquarie University culture studies senior lecturer Chris Muller says social media algorithms reward content with high engagement, which is often dangerous, controversial or harmful. “Algorithms are geared towards keeping us on the sites as long as we can, and they come with a whole set of incentive structures,” he says.
“Content becoming extreme can be a consequence of algorithms incentivising certain types of content on a platform.”
Algorithms fuelled the rise of Andrew Tate, a former kickboxer who amassed millions of followers and 12 billion views with his advice to young men to stop feeling downtrodden, fight back against “feminazis” and control their “bitches”.
Diabetics were denied Ozempic, a lifesaving medication that helps lower blood sugar, after TikTok influencers spruiking its success for weight loss led to a worldwide shortage. Injected into the abdomen once a week, Ozempic was touted across social media platforms as a miracle weight-loss drug, and millions have shared their #ozempicjourney.
Muller says algorithms cannot operate on their own but form part of a “whole technological infrastructure” that “needs to be built up to have an impact”. “They need data, so they are connected to technology to generate data,” he says. “If you think of a typical social media site, this is stuff like likes, scrolls, comments, watch time.”
So, in what appears to be a chicken or the egg situation, algorithms need to be fed data to be built. Is it the user’s fault for being engaged by extreme content, the algorithm’s fault for prioritising it or something else altogether?
In recent years rumours have swirled about the use of overseas bots or troll farms manipulating social media algorithms to sway debate and influence public opinion.
Following the high-profile defamation trial between Johnny Depp and Amber Heard – in which the public was seemingly stacked against Heard – disinformation experts uncovered a co-ordinated campaign of bots posting identical messages attacking Heard. Speaking to the podcast Who Trolled Amber, the experts found more than half of the messages criticising Heard or praising Depp came from inauthentic accounts.
The same distortion, some say, has occurred during the rollout of the It Ends with Us press tour. A deep rift has split those involved in the film adaptation of the popular Colleen Hoover novel of the same name, with Hoover, leading lady Blake Lively and a group of female cast members on one side, and lead actor and director Justin Baldoni on the other.
Fans of the book, which centres on domestic and family violence, have panned Lively online for failing to take the themes seriously when promoting the film, while praising Baldoni for nailing the tone and sentiment. But when Baldoni hired the same PR company used by Depp during his trial against Heard, questions were raised over how many of those negative online comments came from human accounts.
Closer to home, during Liberal senator Linda Reynolds’s mammoth defamation case against her former staff member Brittany Higgins, legal arguments were briefly interrupted for a discussion about an infestation of overseas bots posting content about the trial.
Martin Bennett, representing Reynolds, submitted a 15-page document of X posts that he said showed signs of being published by bot accounts from overseas. He asked the judge hearing the trial, Paul Tottle, to make an order warning about possible contempt.
At least one message highlighted by Bennett was critical of Tottle’s decision to allow Reynolds’s former chief of staff Fiona Brown to give written evidence rather than testify in person because of ongoing ill health.
“Any attempt to interfere with the administration of justice in this case by intimidating, frightening or seeking to deter a witness or anyone else involved in the administration of justice in the context of this case will be punishable as a contempt of court,” Tottle said last month. “To state the obvious, all procedural and substantive decisions in this court are made by me. I do not follow … X and any attempts to influence me by making publications or posts on that platform will be an exercise of futility.” Reynolds, of course, has denied any link to the bot accounts.
Muller says algorithms “can be manipulated in all sorts of ways” including through “bot” or “automatically generated” content weaponised by companies to target individuals with certain posts. It is something to keep an eye on particularly ahead of big elections.
“That’s the whole shady landscape of gaming metrics and that kind of thing. The manipulation of reactions to content that make that content more visible,” Muller says.
“Having some troll farm that will post hundreds of comments on a platform that … might signal to the algorithm, ‘a lot of people are engaging with this, this is highly good for our platform’, because the goal of the platform is to keep users on their site as long as possible, which is how platforms make money because they sell our screen time to advertisers.
“Then any content that can generate those kind of reactions is going to be pushed because it will keep people engaged and it will create that drama that people can connect to when they’re online.”
In the case of Anderson and the blackout challenge, Muller says it’s the first time he has heard a social media platform could be held legally accountable for its algorithm.
While he says it won’t be the last time big tech is seen in court, he says it is unlikely the “blackbox” of social media algorithms will become public any time soon.
“It would be good to have more transparency, the problem is that because the algorithms sit right at the heart of those companies they are really secretly guarded because if they were public … the argument would be that if they shared their corporate secret they would become subject to other people copying their platform,” he says.
“That is one of the reasons it is protected knowledge.”
More Coverage
