The simple post that sparked a barrage of death threats – and the Aussie start-up aiming to end online hate and trolling

Lauren Dubois shared a poignant image online that saw her inundated with graphic threats. An Aussie start-up is aiming to stop that kind of thing happening.

“Lauren, you need to check your Facebook.”

Those seven words still haunt former journalist turned author and content creator Lauren Dubois, who was doing her groceries almost a decade ago when a panicked friend delivered that warning.

Ms Dubois had posted an image on social media of a Muslim bride laying her wedding bouquet at a floral memorial outside the Lindt Cafe in Sydney’s Martin Place.

The emotional newlywed was one of many who travelled to the city centre to grieve and pay tribute to those killed and injured in a terrorist’s siege in December 2014.

The harmless post Lauren shared on Facebook that sparked a barrage of abuse and death threats.

Her simple sharing of a poignant image, meant to highlight the coming together of Sydneysiders in love and peace, in the face of hate and murder, earned her a barrage of abuse and threats.

“My stomach dropped as I looked at the most intense and awful stuff being sent to me,” Ms Dubois recalled.

“I sat in the car and I was shaking. It was people threatening to kill me, threatening my family, all of this really racist, aggressive and violent stuff. It was out of control.”

That was one of her first particularly serious experiences with internet trolls and online hate, but it was definitely not the last.

Author and content creator Lauren Dubois has copped her fair share of abuse and hate online, including some particularly threatening messages.

Those kinds of horrific and dangerous scenarios are what prompted an Australian tech start-up to delve into the darkest corners of the internet to find a solution.

They’re now preparing to rollout a product that just might silence trolls and haters for good.

Having been an online figure for a while now, starting out as a blogger and then moving into the social media space, where she has 200,000 followers on Instagram and another 100,000 on TikTok, Ms Dubois is used to upsetting people.

“It comes with the territory and you can never please everyone, I’ve discovered.

“The funniest exchange was when I put up a meme about how childhood is putting back the $8.60 packet of chicken at the shops because you’ve spotted one for $8.50. Pretty harmless, a bit funny, right?

“This woman abused me, absolutely tore me apart, for promoting factory farmed chicken. I was like, it’s a joke. It’s just a meme. Who said anything about factory farmed chicken? What the hell?”

A lot of the hate she cops is “the usual stuff women get”, from sexist remarks to criticism of her hair, make-up and body.

In the beginning, some of the particularly nasty stuff she copped got under her skin and would leave her feeling rattled.

“I couldn’t get it out of my head sometimes,” Ms Dubois recalled.

“Eventually, I developed a think skin and I had to learn how to change my mindset. People are going to say whatever they want, but it doesn’t really change my life. If I’m existing in their head, that’s their problem. I had to stop letting them live in my head.”

But the occasional death threats and hurtful or aggressive comments about her children still bother her.

Social media can be a cesspit for many, with abuse, hateful content and trolling common – especially for women. Picture: NCA NewsWire / Nicholas Eagar

More recently, her social media comments on the mass deaths of Palestinian civilians in Gaza has attracted a new and sophisticated kind of attack.

“For many years now, when it comes to people abusing me or sending nasty messages or whatever, it usually stops there,” she said.

“These people have taken it a step further where they’ve literally contacted brands that I’ve worked with, and those brands have cancelled contracts with me because they’re getting abused. So, it’s ventured into real-world and real-life consequences.

“That’s the most sinister and upsetting part of it. I can handle people swearing at me or attacking me for my looks or whatever, but when it comes to them actually pursuing me in real life, then it becomes really unnerving.”

Cam McMaster has heard countless stories over the past several months while developing the new algorithm and AI-powered app Recant.

“Recant is a technology that can apply filters to social media user accounts that applies parameters around what they deem to be inappropriate messages,” Mr McMaster explained.

“When it detects content that violates their boundaries, Recant removes it from view to preserve their sanity and their online sanctuary. It also responds accordingly to unsafe content, with various referral mechanisms involving police and the platforms themselves.”

A demo of the Recant technology, currently in beta testing phase. Picture: Recant AI

Filtering works on a temperature scale from one to 10 covering all kinds of topics, from sex and sexuality to race and diversity.

“That allows Recant to account for personal context,” Mr McMaster explained.

“Say you’re a comedian who gets messages that might otherwise be edgy or alarming for someone not in that space, but for them, they’re expected in the context of their world.”

Or perhaps it’s a young person who’s single and ready to mingle but draws the line at an unsolicited nude photograph.

The idea was inspired by a screenshot Mr McMaster saw of a woman’s inbox on LinkedIn, a space that should be the cleanest given its professional nature.

“It was a sewer,” he recalled. “Absolutely disgusting stuff from men sending her vile and sexual messages. Our thought was that the people developing and running this social media platforms aren’t doing enough in terms of safety and security, so there was a big gap there.”

Author and content creator Lauren Dubois has copped her fair share of abuse and hate online, including some particularly threatening messages.

Aside from services that allow parents to monitor or lockdown the devices and online accounts of children, and very basic platform moderation plug-ins for big brands, there aren’t any comparable products in the market.

That surprises Mr McMaster, given how prevalent online abuse has become – particularly towards women.

“It just seems bizarre to us that grown men are perpetrating such atrocious behaviour with absolutely no consequences. It seems kind of irresponsible on the parts of the platforms,” he said.

“And when we’re having conversations with beta testers about Recant, they all ask the same thing: ‘Why aren’t the platforms doing this?’

“It’s a good question, because they probably could. But I suspect that if they were, it would set a precedent in terms of moderating stuff and holding users to account for their horrid behaviour, and that costs a lot of money for little return.”

Originally published as The simple post that sparked a barrage of death threats – and the Aussie start-up aiming to end online hate and trolling