RealThing brings AI to its RealSAM phones for the blind

Melbourne start-up RealThing Ai is using AI to enhance its modified smartphones that read news to blind and partially sighted people.

RealThing Ai is building an enhanced version of its RealSAM phone with AI for the visually impaired.

Melbourne start-up RealThing achieved international success with its modified smartphones that read news from a hundred sources to blind and partially sighted people.

Now it is deploying artificial intelligence to add a range of new features. CEO Nick Howden told The Australian that the start-up. which also employs former military and aerospace staff on its AI effort, had recently received a $1 million grant from The Australian Government. RealThing, now RealThing Ai has written contracts with organisations helping the visually impaired in the UK and US.

The company began developing its technology about eight years ago. The Australian reported on the very early trials that took place in 2014.

After initial testing in Australia, several thousand RealSAM handsets were rolled out in the UK from 2017 through the Royal National Institute of Blind People.

Mr Howden said RealThing Ai has developed a partnership with Vision Australia and has contracted with the US Library of Congress National Library Service to supply RealSAM units there.

He said RealSAM was now “a Swiss Army knife” for visually impaired people which reads news, magazines, and books, and plays podcasts and the radio.

He said the handset also linked users with the Be My Eyes service, where a blind person can “borrow a sighted person for a few minutes” to describe the view through the user’s smartphone camera.

RealThing Ai CEO Nick Howden

Mr Howden said there were about three million volunteers globally who supported Be My Eyes.

“You can see out this person‘s camera, and they can point it at something and say, is that dog food or baked beans? There’s plenty of volunteers, the issue has been getting blind people to use it, so we’ve provided a really simple voice interface. They can pick up our device and say ‘be my eyes’ and it connects them.

“Our goal is to provide a whole pocket full of services and control it all with your voice.

“You can turn on location services and it‘ll tell you when you’re walking to the local shops, where you are and how far away it is.”

He said this feature had required tapping into various mapping and location services. The key was having all services combined on to one device, and accessed with voice.

He said the new version of RealSAM was being tested in Australia through an agreement with Vision Australia. A launch of this augmented product would take place in Australia later this year.

He said the US Library of Congress’s National Library Service had wanted to replace their current technology which comprised about half a million cartridge devices in the community. “We’re now replacing their reading devices with ours.”

He said RealThing Ai had been developing an AI engine that offered intelligent dialogue so that users could have a conversation with their RealSAM unit.

RealThing Ai’s RealSAM phone for the visually impaired.

“Most voice interfaces like Siri and Amazon Alexa are pretty good at single utterances, so saying ‘play me a song by David Bowie’ is fine. As soon as you want to do something a little more complex, they actually can’t deal with it.”

He said when reading a newspaper, a multi-step approach would let a blind reader start reading, ask about available headlines, select one, read, back out and go to another story, all controlled by a natural, ongoing long term conversation.

The AI engine was being built on the BDI (belief-desire-intention) concept from aerospace and defence which involved modelling human behaviour in simulation. “We came to the realisation that this modelling is very good at human dialogue.”

Mr Howden said the grant money would help RealThing Ai convert the dialogue software to work on phones without internet support. It currently worked in the cloud.

“Essentially we‘re moving all the AI onto the handset, and we’re building an enterprise product which is specifically targeting groups like Library of Congress and Veterans Affairs and these big organisations that might have hundreds of thousands of clients across the country.”

He said RealThing Ai employed 18 staff but expected to grow significantly in the next two years. Staff were located in Melbourne, the UK, US and Italy.

He said the funding would lead to the creation of jobs in the high value areas of AI and MedTech.

Industry, Science and Technology minister Karen Andrews said Australian businesses were among the most innovative in the world and the government was pleased to help scale-up their ideas.

“The Accelerating Commercialisation grants are all about investing in the growth of great Australian business ideas that will ultimately change our lives for the better.”