YouTube’s algorithms recommending ‘incel’, ‘manosphere’ videos

An Australian internet advocacy group is calling for lawmakers to force social media platforms to share details about their algorithms after its research found YouTube’s recommendation engine is pushing men towards misogynist content.

Reset Australia, which is privately funded and has been backed by eBay founder Pierre Omidyar and mining billionaire Andrew Forrest, conducted a study which it says suggests YouTube is directing male users towards anti-feminist content and trapping some men in far-right internet bubbles it describes as a ‘manosphere’.

The group, which is critical of social media, created 10 accounts on the dominant video sharing service designed to mimic young men and boys aged between 13 and 20.

Reset Australia argues YouTube should be forced to be more transparent about its algorithms and be held accountable for a wider array of harm.Credit: Getty Images

Eight of the accounts, which were run by analysts from the UK anti-extremist think tank the Institute for Strategic Dialogue, were set up to follow generalist right-wing accounts ranging from United Australia Party leader Craig Kelly to the fringe right-wing extremist Blair Cottrell. Two were left blank.

Within hours of watching and liking videos recommended by YouTube on both its primary product and YouTube Shorts, which is focused on shorter clips similar to TikTok, all of the accounts were served up anti-feminist clips. In the fortnight-long study others were later also shown clips of music videos in honour of fascist leaders.

“This is yet another example of the business model of big tech placing profit ahead of safety – women’s safety,” said Reset board director Catriona Wallace. “We need to urgently put in place regulation that forces social media platforms to be transparent about the risks of their algorithms and redesign how they promote content so that they align with the feminist futures we want to create.”

The federal government has led a worldwide crackdown on big technology companies in recent years, introducing tough regulations for livestreaming of violent videos, threatening platforms with defamation liability and establishing the media bargaining code.

But Reset argues it ought to go further by focusing on the algorithms that power the platforms. Otherwise, Reset said, social media risked serving as a pipeline directing men and boys into extreme forms of misogyny that can escalate to violence.

The study is small scale and its findings do not include a statistical analysis of the videos recommended to the accounts. The test accounts under 18 were mostly shown age appropriate content on YouTube proper, but were shown the same content as older accounts on YouTube Shorts. The platform has parental supervision tools for younger users.

A YouTube spokeswoman said the company could not comment specifically on the study without seeing its finding and methodology, but pointed to the site’s community guidelines that prohibit violent extremism and hate speech. In the final quarter of last year it removed 3.8 million videos, the spokeswoman said.

“Beyond removing content, since 2019 we’ve also limited the reach of content that does not violate our policies but brushes up against the line, by making sure our systems are not widely recommending it to those not seeking it,” the spokeswoman said. “We welcome more research on this front, but as other researchers have noted, our systems often point to authoritative content.”

The videos that the accounts were recommended appears to fall just inside YouTube’s boundaries for people over 18.

Jordan Petersen, the Canadian emeritus professor and motivational speaker, was a constant in the recommendations with clips of him “putting a feminist in her place” shown to the test accounts. He was a gateway to “manosphere” content, the report said, referring to a collection of movements that tap into men who feel alienated from a society they see as being too good to women.

Hunter Johnson, the chief executive of emotional intelligence charity The Man Cave, said the findings were alarming but not unsurprising. Young men had been left feeling isolated, confused and disengaged by the pandemic, Johnson said, and society needed to work to re-engage them.

Reset found YouTube pushed similar content to the accounts that were already following right-wing influencers. “The content YouTube provided appeared to stay within the same alt-right bubble,” the report reads. “However, in the case of the ‘blank’ control accounts which did not engage with seed list content creators and instead purely followed YouTube’s recommendations, the algorithmic recommendations did appear to direct the accounts down an increasingly concerning path.”

The two test accounts were shown videos about “sigma males”, a fringe term for men who are separate to society but sexually and socially popular, and “incels” or men who are unable to have sex because they cannot find a partner.

In a blog post last year, YouTube’s vice president of engineering Cristos Goodrow, said that videos YouTube defines as “borderline” were demoted by the site’s recommendation systems. “Consumption of news and political content on YouTube more generally reflects personal preferences that can be seen across their online habits,” he said.