As it happened: Quest 3, AI agents, smart glasses revealed at Meta Connect

After some final words about creating Meta’s new AI agents, and particularly about the open source elements that let people analyse what’s going on under the hood, that’s all for the Connect keynote. Thanks so much for following along, and look out for articles today and over the rest of the week with first looks at Quest 3 and more local details.

Li-Chen Miller, Meta’s head of smart glasses is here to talk about the new specs. It has ultra-wide 12MP cameras and can take 1080p vertical video. A new processor powers advanced stabilisation, there’s 32GB of internal memory for storing clips and videos, and an array of five microphones capture spatial audio in videos while isolating your voice in calls.

The custom speakers have been completely redesigned with twice the bass and reduced audio leakage.

Because the Meta AI lives on the glasses, you can say something like “send a photo to Heidi”, and it will take a photo and send it to your contact in the app you usually talk to them, without you needing to take out your phone.

The specs come in Wayfarer and retro Headliner styles, with an option for coloured transparent frames or classic black. You can also customise the frame and lens colours for more than 150 combinations. The glasses come in a case that charges the unit, for up to 36 hours of use.

For live streams, you start a stream as usual and then tap a glasses icon to switch to your point of view. Tapping and holding the side of glasses will read out the most recent comment into your ear.

Li-Chen Miller rocks the new smart glasses.Credit: Tim Biggs

Melissa Brown, Meta’s head of developer relations, has taken the stage to talk about building games in mixed reality. The entire existing library of Quest games will look and play better in Quest 3, she says, as developers push out updates that take advantage of the new tech.

Xbox Cloud gaming is coming to Quest 3 in December, and the headset will work with a new subscription called Quest+ which gives players two free games every month.

Meta is also rolling out a lot of new content to its Horizon Worlds social space, including many more options for creating your virtual avatar.

Brown also shows off immersive video, like live sports and concerts, which are streamed in 360 degrees so you get the same view you’d get actually being there.

Zuckerberg finishes up by reiterating that Meta plans to make all its technologies as accessible and affordable as possible, before handing the stage to chief technology officer Andrew Bosworth.

Bosworth is breaking down some of the technical innovations in Quest 3. You no longer need to manually set a guardian boundary, since it can scan your surroundings and warn you if you’re too close to objects. You can tap your headset any time to switch between fully immersive VR and mixed reality. A demo shows a bedroom where playlists from iHeartRadio live on walls like rock music posters, and the roof has been replaced by a starscape.

He calls Quest 3 the “best value spatial computing” there will be for a long time, taking a clear shot at Apple’s marketing speak and its upcoming headset’s $US3000 price tag.

A new generation of Meta and Ray-Ban’s smart glasses have been revealed, and they’ll have Meta AI built in. So in addition to taking pictures and video from your face, you’ll have access to an AI in your ear at all times like Iron Man.

Coming next year, a free software update will also let the AI understand what you’re looking at, so you can look at a sign in Spanish and ask what it says, or ask what a certain landmark or artwork is. Zuckerberg says glasses are the most obvious format for letting an AI see what you see and hear what you hear. He thinks it will let smart glasses take off long before the technology exists to shrink VR and AR goggles into a glasses format.

The new glasses will also allow users to stream video live from their point of view. They’re coming to the US on October 17 for $US299.

Ray-Bans Meta Smart Glasses put an AI in your ear.Credit: Tim Biggs

Zuckerberg has stressed that there will be a lot of challenges in build AI responsibly, from making sure the AIs of businesses and creators reflect their values, to blocking inappropriate content. Meta is rolling the tech out more slowly than it usually would to address issues.

“This is going to transform how people use all our products,” he says.

Meta AI is the name of the company’s basic assistant, which can get real-time information from Bing or talk to Emu to generate images on the fly. It will be available across Meta’s products. But there are also agents for more specific helpers.

Max is a chef who can help you with your cooking. Lily is a creative writing partner. There’s also a travel expert, an MMA expert, a singer-songwriter, a “hype woman bestie” and more. All of these AI agents have realistic-looking profile photos, so talking to them is designed to be just like talking to friends. The photos are not AI generated, the agents are played by real people and have short animations showing their reactions to your queries.

Zuckerberg cracks up while showing an AI agent called Dungeon Master who is played by Snoop Dogg. He types in real time for a bit, showing that the AI’s responses are very quick. Mr Beast, Paris Hilton and other celebrities also show up as agents.

In the future developers will be able to create their own agents, who could show up in the Metaverse or be used by businesses for chat support.

Snoop Dogg appears as an AI RPG master.Credit: Tim Biggs

Zuckerberg says Meta will integrate generative AI into the apps billions of people already use. He also unveils Emu, the company’s new image generation engine that can create pictures based on text prompts in as little as five seconds. In WhatsApp for example, you can generate new stickers or anything to share in chats.

In around a month, Emu will also come to Instagram to allow you to change images instantly. An example showed Zuckerberg’s dog becoming a cross-stitch, or having its dreadlock-like coat straightened.

Meta is also creating a platform for building different AI agents. They’ll have profiles on Facebook and Instagram, and will eventually have presences in the multiverse. Meta has already built a bunch of them.

Meta is doubling down on AI.Credit: Tim Biggs

If you don’t want your real life interfering with your virtual activities, the Quest 2 will of course support the full range of fully immersive content already on the store, plus loads more to come.

The device is the first to be powered by the Snapdragon XR2 from Qualcomm, allowing graphics intensive games like Asgard’s Wrath II and Assassin’s Creed Nexus to run natively. Roblox is launching on Quest today.

In the US, the device is shipping on October 10 for $US499.

Meta Quest for Business is also coming soon, giving workplaces access to mixed reality offices through the Quest 3.

Zuckerberg says the Quest 3 is the first mainstream device that can blend the physical and digital worlds together. The most powerful headset Meta has ever created, it’s designed to expand the possibilities of your physical space.

Quest 3 displays your physical space with 10x the pixels as the Quest 2, Zuckerberg says, and automatically scans the world around you so for example you can throw a virtual ball and have it hit the real wall, or duck behind a real couch to hide from pretend aliens.

You can also create a massive virtual screen anywhere you go to watch movies, live sports, or games streamed from the Xbox cloud. Later on, Meta will let you place virtual objects around your home which will stay in the same place every time you put the headset on.