Meta CEO Mark Zuckerberg outlines AI research for metaverse development

Meta recently hosted a live-streamed event called “Inside the Lab” on May 4th, during which CEO Mark Zuckerberg discussed the company’s AI research efforts that are essential for constructing the metaverse – a virtual space for working, socializing, and playing. Zuckerberg introduced the concept of Builder Bot, a new type of generative AI model that enables users to describe a world and create aspects of it. Zuckerberg believes that as technology advances, users will be able to create complex environments with their voice to share and experience with others.

In addition, Meta is conducting AI research to enable individuals to have more seamless interactions with voice assistants, a crucial stride in terms of how people will communicate with AI in the metaverse. Project CAIRaoke is an AI-based on-device assistant that Meta is developing for cooking purposes, where the voice assistant can chime in to warn when salt has already been added to the pot, and notice when they are running low on salt and order more. Meta is currently using this model in their Portal video-calling device and plans to integrate it into devices that support AR and VR. Meta is also working on a universal speech translator that will enable instant speech-to-speech translation across all languages.

Meta is investigating the potential applications of AI in moderating content and activities in the metaverse, according to Jérôme Pesenti, the head of AI at Meta. The company already uses AI for moderation on its existing platforms, but the metaverse will require more real-time moderation, and the policies for metaverse activity are still being developed. To anticipate the types of interactions that could occur in the metaverse, Meta is exploring “self-supervised learning,” where AI is given unprocessed data instead of being trained on labeled data, and egocentric data, which entails viewing worlds from a first-person perspective. Meta has formed a global consortium of 13 universities and labs to work on the Ego4D dataset, the most comprehensive egocentric dataset. Meta also plans to extend free education initiatives to increase the representation of racial minorities in tech, which is essential for creating unbiased AI systems.

Meta plans to make TorchRec, an open-source recommendation library used to personalize products like Facebook’s news feed, and publish a feed ranking prototype that shows how its algorithms prioritize content for Instagram users. Pesenti said in an interview with Reuters that Meta is tightly controlling the responses of its new CAIRaoke-based assistant until it can ensure that the system does not generate offensive language, as language models are very powerful. Overall, Meta is working on various AI research projects to improve the user experience in the metaverse, including creating nuanced worlds using voice commands, more natural interactions with voice assistants, content moderation, and expanding free education initiatives to create unbiased AI systems.