Meta Connect 2024: Everything Revealed in 12 Minutes

This is the new MetaQuest 3S. High quality mixed reality for $299. Hell yeah. I've been waiting for this one for a long time. You know, when we built Quest 3, we delivered the first high quality mainstream mixed reality experience, where you can see the world around you. You can reach out and touch digital things as if they're right there. Quest 3 is the best mixed reality device that you can buy today, and I am really proud of it. Quest 3S has the same defining features as Quest 3. High resolution, color, mixed reality, powered by the same processor, vivid pass-through, hand tracking, feels natural, Touch Plus controllers for precision. You get the full range of experiences from Horizon OS, gaming, social, fitness, watching videos, productivity, and more. So whether you're looking to get started with the base Quest 3S or if you want the top of the line Quest 3 with 512 gigabytes and 4K displays, which is now just $499, the Quest 3 family, it's not just the best value, it is the best product available. Everything you're going to see up here is streamed from his headset live. This is what he is seeing. Yeah, it's sharp. All right, Kenny, what do we got? All right, now the first thing that you can see is that we have expanded the quality, the volume of content massively. Yeah, sorry to the dude next to him. Live demo artifact. All right, you can now run everything from 2D mobile apps to remote desktop for PC to fully immersive experiences all on one device. You can open screens. You can put them anywhere around you. You can have pretty much as many of them running as you want. And we have also rebuilt all of our social apps for mixed reality too. We've got all new Instagram and Facebook experiences to bring your feeds and your reels with you into mixed reality. A really nice interface so you can lean back, enjoy your content on nice big screens. And there's a bunch more too. We've been working with Microsoft to upgrade remote desktop soon. So soon you're going to be able to easily connect to any Windows 11 PC. You just look at the keyboard, and it'll start pairing. So if you want to work on a giant virtual display or a bunch of virtual displays, or if you want to work on your projects without distractions or with other people, Quest is going to be able to be a natural extension of your PC. We're also bringing photorealistic spaces to the metaverse. And we call this Hyperscape. And the way this works is you can use your phone to scan a room and then recreate it. Or you can step into a room that someone else has scanned and shared. Now, of course, there's gaming. So we've got some really good ones coming, showing off what the Quest 3 family can do. Batman, Arkham Shadow up here. Exclusively on the Quest 3 family, it's going to be included in any purchase of a Quest 3 or 3S that you buy this fall. Lots of great experiences, a lot more coming soon. Today, we are making meta AI even smarter with our new open source LLAMA 3.2 models. And I'm going to get into more details about LLAMA 3.2 in just a minute. But one of the headlines is that they're multimodal. So now meta AI can natively understand images as well as text. Now, we're using this new capability to build some features that I haven't seen anyone else build. I mean, this is pretty interesting and novel stuff. Creative tools that we're adding to Imagine Edit. It lets you upload any photo and edit it precisely with natural language in meta AI across the apps. I think that voice is going to be a way more natural way of interacting with AI than text. And probably, I think it has the potential to be one of, if not the most frequent ways that we all interact with AI. It is just a lot better. So now you're going to be able to have natural voice conversations with meta AI across all the major apps. You've got Instagram, WhatsApp, Messenger, Facebook. Hey, are live demos risky? Live demos can be risky, yes. They can be unpredictable, prone to technical issues, and potentially embarrassing. However, they can also be engaging, memorable. But I'm going to do it anyway. Go for it. Just be prepared for anything to happen. So you were one of the first creators to build one of these creator AIs with a text-only platform. Can you tell us what it does and your experience with it? Yeah, so for me, I've been treating my creator AI kind of like my digital assistant. So I've trained it on how I respond and how to engage with my audience. And then, yeah, I can send DMs, can share links to relevant resources. And then, what else am I having it do? Oh, yeah, it guides people through conversation. It's great. Congrats on the new book that you just released. What's the main thing that you're hoping that people take away from it? Thank you so much. Yeah, the main thing I want people to take away from my book is the idea that you have the power to create your own opportunities by combining curiosity, adaptability, and resilience in a rapidly evolving digital world. We are experimenting with automatic video dubbing on Reels, starting with English and Spanish. So this is going to be a way that you can see more of the content that's out there, no matter what language you speak. And as a creator, this is going to let you reach a lot more people around the world, no matter what language you speak. So now, you're going to be able to take your content, and it's going to be in your authentic voice, speaking other languages, if you want. And we're also doing automatic lip syncing. So that way, it actually looks like you are speaking the other language. So let's check this one out. Mi lengua materna es el español, pero fíjate, ahora también puedo hablar inglés a través del doblaje automático de IA. My native language is Spanish, but check it out. Now I can also speak English through AI automatic dubbing. So glasses, they're kind of the perfect form factor for AI, right, for letting an AI assistant see what you see, hear what you hear, be able to communicate with you privately, so it can help you out with whatever you're doing throughout the day. I am about to have a conversation with Brandon Moreno, former UFC flyweight champion of the world, where he is speaking Spanish, and I am speaking English. All right, Brandon, you want to come out here and show us this? Starting live translation from Spanish to English. Hey, ¿cómo estás? ¿Cómo está tu rodilla? ¿Has vuelto a entrenar? Hey, how are you? How is your knee back training? Doing well. Knee is doing well. It feels strong. I'm ramping up training. How are you doing? Are you getting ready for your next fight? Haciendo bien. La rodilla está. Está haciendo bien y sentirse fuerte. So today, we are dropping a limited edition of clear, transparent frames. And I think this is neat, because it's a neat vibe. You can see the technology in them. For the first few versions, we spent so much energy designing the glasses to basically hide the technology, to make them look like normal, stylish glasses. But now I think people are starting to appreciate what makes them special. And I think having a design that celebrates that is pretty awesome. This is Orion, our first fully functioning prototype. And if I do say so, the most advanced glasses the world has ever seen. They're not a headset, no wires, less than 100 grams. They need wide field of view, holographic displays, sharp enough to pick up details, bright enough to see in different lighting conditions, large enough to display a cinema screen or multiple monitors for working wherever you go, whether you're in a coffee shop or on a plane or wherever you are. And you need to be able to see through them. And people need to be able to see through them too and make eye contact with you. This isn't pass-through. This is the physical world with holograms overlaid on it. It is a completely new kind of display architecture with these tiny projectors in the arms of the glasses that shoot light into wave guides that have nanoscale 3D structures etched into the lenses so they can diffract light and put holograms at different depths and sizes into the world in front of you. And all of that is directed by custom silicon and sensors that we designed and powered by a battery that fits in the arm of the glasses. It is a absolutely incredible amount of technology to be able to miniaturize and fit into a pair of glasses and a small puck that goes with it to help power the whole thing. They're going to do voice and AI. They're going to do hand tracking and eye tracking so you can select UI elements by looking at them. Voice is great, but the thing is sometimes you're in public and you don't want to say what you're trying to do with your computer out loud. Hand tracking is neat for controlling different interfaces, but you don't want to walk down the street like this. So I think that you need a device that allows you to just send a signal from your brain to the device. So this isn't just the first full screen, full wide field of view holographic AR glasses. This is also the first device that is powered by our wrist-based neural interface. We do still have a few things that I want us to keep pushing on before we ship this as a consumer product. We're going to keep tuning the display system to make it sharper. I want to keep working on the design to make it smaller and a bit more fashionable. And we need to keep working on the manufacturing to make it a lot more affordable too. But we have line of sight to all of those things. So we are going to use Orion as a dev kit. We are going to use it mostly internally to build out the software that we need to, but we're also going to work with a handful of partners externally to make sure that we get a diversity of content and just that we can really dial in the software and the experience. So that way, when we have the next version of this hardware, it is going to be ready to be our first consumer full holographic AR glasses.