Accelerating the Spatial Computing Revolution for Developers and Enterprise with Ziad Asghar

All right. You know, this is truly a great time to be working in XR with all the great advancements that we see from a technology perspective, from IP to all the amazing use cases and experiences that our partners are building. I truly agree with AI that this is the best time for XR, and this is the time when I think the pace of innovation that we will see is going to be far faster than what we have seen in the last few years. Right? This is that time now as we have seen the different transitions from the computing space to the mobile computing world and then onto the spatial computing side. You know, we have seen how the technology has evolved, but I think the time is right. The technology is ready for spatial computing at this point in time. The challenges are there too, right? If you really look at it, what you will notice is that basically we need to bring the same level of compute that we have had on devices that are like MR devices into much, much lighter devices today. This is smart glasses. These are AI glasses. And the challenge that we have is we have to do it in like a tenth of the power. We have to do it in a thirtieth of the size that we used to. This is what I call a Qualcomm-sized problem. But what gives me a lot of confidence in this is that basically it's the advent of AI, and I think Ori talked about it too. But XR and AI are two technologies that are truly symbiotic. They're working together, and many of the challenges that we have seen from the perspective of XR, for example, content creation on the MR side. Well, what does a great job of creating content? Well, generative AI working with XR can absolutely do that. The ability to be able to have agents that are running on a small device like a smart glass or an AI glass can totally change the way that we interact with technology. I'll even go as far as saying that the best interface for agentic AI is a smart glass. So, we're very focused on solving exactly these problems.

Now, the approach that we have taken to take on this challenge is really to be able to focus at every different level. We're driving the transistor. We're going to smaller and smaller transistors, which mean we can do more work for any given amount of power. I have personally been focused on driving better camera technology, better AI technology that allows you to do more AI inferencing for a given amount of power than anybody else out there. We can do amazing computer vision algorithms. And then what we have also done as a company over the last one and a half decades is to really build up the perception stack. These are technologies that allow you to interact with the virtual world. Hand tracking and eye tracking and SLAM and all those technologies. And you know, the things that we have learned from those, we've actually taken those learnings and actually hardened them in silicon. And by virtue of that, we can do all of those things at a power point that's far better than what anybody else can do. We have the most elaborate roadmap today, and I'll actually be sharing some additional new members of this roadmap today. So very excited about that. But also, I really want to thank all of our Snapdragon Spaces developers as well, and of course, working with Android XR. I'll show you the path as to how we're going to be transitioning all the amazing work that you have done.

Now, the other innovation vector that I see is that in the past it used to be just a single MR-like device. MR device with, you know, VR only, so no video see-through. We have transitioned now to MR with full video see-through technology. Now we're heading towards more devices that are more optical see-through, and you can actually offload some of the compute and AI to a companion device to be able to make the device lighter. This has been one of the challenges, right? How to make the device more ergonomically easier to use? How can you keep on wearing it for the longest time? Now there are some devices that you can wear and, you know, you build very strong neck muscles. But I don't know if we all want to do that. But nonetheless, we have the different innovation that's coming up now in this space to be able to truly allow our partners to differentiate even at the form factor level. Now, similarly, in augmented reality and AI glasses side, we have great partners like Meta that have taken this great platform and they're actually offloading the AI to the cloud by virtue of going through the smartphone. We also have the next collaboration coming in which now offloads the AI and the smart glass capability onto a smartphone. In the future, we'll see that happen on the PC, on the car. All of those are platforms that have a lot of compute in them today that we can offload a light device like smart glass to. And then we're also seeing multiple partners that actually are creating a compute block like the puck that you see over there, and they're using that as a way to be able to do it. So amazing amount of room for all of us to innovate and truly differentiate not just on the AI experiences but overall XR experiences and then even on the form factor side.

Now, these are just some of the devices that we have launched since the last AWE. You will notice they are really every form factor possible. These are devices that are light, like smart glasses and AI glasses, but the best-in-class devices like the Quest 3S that are fully immersive with amazing video see-through technology. And you know, really we are focused on making sure that our platforms allow people to bring in all of these different form factors to the consumers.

I want to start with the lightest first, right? As we talk about these different form factors, the Meta Ray-Ban truly stands out, right? It's really a breakout success. This is a device that's fashionable. It's able to allow you to get and take this amazing image quality and the pristine camera technology that we have to be able to take that and do proper multimodal AI. Right? Multimodality is one of the promises of what we can do with AI and XR today. It uses the amazing sensing hub that we have in the Snapdragon AR1 platform to be able to do this processing at the lowest power possible. And with that, they've really created an amazing, amazing experience. To talk more about what's coming up next with Meta Ray-Bans, I'd like to invite my friend Bruno Sanden from Meta. Bruno.

Thank you, Bruno. Amazing to see all this great work that's going on in the space. So very excited about this engagement that we have with Meta as well. Now, continuing on with these amazing form factors that we are working with our partners on is Snap Spectacles. A truly differentiated product as well. This is an all-in-one immersive AR device, and I talked about the challenge on the first page. And really, what we have done with Snap over here is a completely different architecture. So this is a dual SoC architecture. One that's focused much more on running applications. The second that's focused much more on running the actual perception stack. And with that, they have created an amazing experience that you're able to see outside. I know Evan will be stepping up to the stage very soon right after me, and he'll have a lot more interesting things to share as well. But it uses all the goodness that we put in from camera technology, from our full Adreno GPU rendering, and the best-in-class picture quality on our displays as well.

I wanted to spend a little bit of time talking about the different verticals that we have, and I think there is so much that we can do. This is one of the experiences that we actually built with our partners at KDDI working with Capcom on the entertainment side, and it truly shows when it's done well, it can really be an amazing, amazing experience. So please take a look. This is only available here at our booth. Only place you'll see it outside of Japan.

So, please check it out in our booth also. Really, really amazing experience. So, continuing on, talking a little bit more about what we see from a verticals perspective, one that I'm truly excited about and would really want to work a lot more on is the education vertical. So we are actually working with our partner Prisms, and they're going to be launching comprehensive grade 6 to 12 math and science immersive education in 300 school districts across 28 states. And what they have seen is almost an 11% efficacy improvement when it comes to increase on benchmarks for Algebra 1 assessment. And by the way, this is something that's closely correlated with future wages for a person, right? So amazing amount of benefit when we combine XR and education. So please check it out in our booth as well. But I want more of us in this room to be working on this vector as well. Great promise here.

Another one that we're working with our partners on is health and wellness. This is an area which actually again is somewhat undersubscribed in my opinion today. But what we have done is working with VA Immersive is that they're actually taking MR technology to be able to help patients, and they've done about 10,000 immersive visits with patients. And what they're seeing is almost a 27% decrease in pain intensity, 28% decrease in anxiety, and almost 50% improvement in subjective units of distress. Right? This is without drugs, by the way. Amazing amount of potential in this space. Again, we want our partners in this space to be able to innovate and really come up with experiences on the health and wellness side, too. And I cannot stop on the verticals without talking about sports. I think this is one that has again massive promise. This is where you get to see a vantage point that you just cannot see anywhere else except in mixed reality. Now, as you guys remember from the last time, we had actually introduced our first cohort for the XR Sports Alliance, right? This was a great set of partners with us. But today, I'm very excited to be announcing the second cohort, and it has really some key hitters. So we got T-Mobile, we got Google, we got Lenovo, Red Bull, and many other key partners. And this is when we start to move towards proof of concept and start to move towards execution. I'm really looking forward to seeing devices with this capability on them. It's very, very exciting.

All right. So now pivoting to the second kind of vector that I talked about where you take a smart glass-like device and now you offload it to the smartphone. We're very excited about the work that we are doing with our partners at Google and Samsung where we are bringing this vast platform that is Android today to the XR space. And I'll also talk about how we then take that and allow our developers to be able to use this amazing platform. To talk more about it, I'd really like to invite my friend Hugo to come on the stage and give us an overview.

Thank you, Hugo. You can see the potential of this collaboration, right? This vast platform that we have with us, which is basically Android, and how we're able to bring that to the XR space along with Android XR on the Moohan-like platform. We also have our partners at XREAL, who I believe Chi will be going on today also on the stage. And this actually uses again our technology as well, working with what XREAL has done on Project Aura. This is a lighter platform with optical see-through, again giving the consumers all the options to be able to really bring all those advantages to the consumers and our partners. Again, it utilizes the six degrees of freedom perception technology and many of the capabilities that we have in our solutions through the wired compute device.

All right. So, continuing on the Android XR theme now, I want to first of all thank all of our great Snapdragon Spaces partners who have done amazing experiences that they have created on Snapdragon Spaces. I want to today talk about how we want to take their investments and the work that they have done and move into the Android XR realm. So, again, I'm super excited today to be announcing the Snapdragon Spaces compatibility plugin for Android XR. It's available now. You can absolutely start to download it and start to work on it. This is, there's going to be a session tomorrow at 1:00 p.m. where we're going to actually work with Unity to be able to show you the full experience as to how you can take your application and your work onto the Android XR platform. And actually, one of our partners, Siru, did that exercise, and in about a couple of days, they were actually able to take their work from Spaces onto Android XR. Very promising and very, very exciting as to what we have coming up next with our developer partners.

All right, so I talked about this slide in the beginning, and if you notice, what I really wanted to spend a little bit of time on was the rightmost part on this slide, which is that we have talked about how we can offload a lot of the work on the AI and the rendering side when we talk about doing AI processing on these light devices. But I also want to talk about how this device itself has the capability to be an individual device that you use just as a standalone device. That means to be able to do certain amount of AI processing on that device. And that's what I want to talk about next. Now, AI is super central when it comes to XR, whether we talk about MR or whether we talk about AR. Now, when you discuss MR, one of the challenges that people have had is the availability of content. But today we can apply techniques like Gaussian splatting and NeRFs and others to be able to bring 2D content into the 3D content realm. We can take models that are basically speech-to-3D, text-to-3D models. We have models like Sora and Stable Diffusion and others that basically enable us to be able to bring amazing content into the MR space. At the same time, you can go into a game and you can say, 'Well, create me this sword,' and GenAI can actually create that for you on the fly. What I envision is basically that the device knows your likes and dislikes and the colors and textures and the themes that you like. And it's actually able to create a world that's suited to you and to you, just each one of you. It's not a world that basically a graphic designer creates in one place and everybody gets to see it, but it is something that's customized to each one of you. And that's the personalized experience that I believe generative AI can bring in MR. Similarly, on the AR side, like I said earlier, you can actually have an agent, an assistant that's sitting on your glass that can help you with fitness, that can help you with navigation, but really what it is is it can actually take full tasks, break it down into smaller pieces, and actually execute them for you. That is the promise of AR with AI. And similarly, if you see on the slide, I think the key part also is the contextual information that we are able to get. We have these amazing technologies within our solutions such as the Snapdragon XR and AR product lines where you can actually take the content, the context, and be able to put it into this sensing hub. And if you look at the context information that you have on a watch versus the context information that you may have on a smartphone or an auto, it's very different. But you can imagine now that if I could combine that context, I can do generative AI far better than what you can do on a single device. What do I mean by that? With a smart glass, it's able to see what I'm able to see. It's able to hear what I hear. With a smartwatch, I know the heart rate, I need the stress level, I know the activity level. If you can combine all of that into a generative AI experience, you really get to a point where basically you can have true personalized hybrid AI. And that's what we're driving the technology towards. Of course, we're working with all of our partners today to be able to make that happen.

All right, so we spent a little bit of time talking about all the devices today. The devices that you see coming up, they're all driven by our platform, the Snapdragon AR1 Gen 1. Now, I'm very, very excited to be talking about a new member of our product roadmap, which is the Snapdragon AR1 Plus Gen 1. And we've actually made this happen with all that we hear from our partners. And we have it launching today. This platform is going to be the world's most advanced solution for AI smart glasses. Why do I say it's the most advanced solution for AI smart glasses? If you look at the key vectors that are important from an AI smart glasses perspective, you need a smaller package. You need it to be able to fit on the side of your glasses. Well, we've reduced that by 20%. We've improved the power by 7% in many of the key use cases. And most importantly, one of the key features is the camera technology. We've brought in basically massive multi-frame engine and motion-compensated temporal filtering to get to the highest-end camera capability. You can imagine you're sitting in a dark restaurant in the night. You have a menu in front of you. You want your smart glasses to be able to read that text on it. That requires this technology. So we brought that into space. But something that I'm really, really excited about is what I said on the previous topic, which was this device can do on-device language models. Can you imagine the session that you do with ChatGPT? You can actually do it just on a smart glass, not connected to the cloud, not connected to a smartphone, not relying on any other device. So we can run a billion-parameter model natively on this smart glass with very good time-to-first-token and about 13 tokens per second, which is actually pretty darn fast. But I don't want to just talk about it. I actually want to show you the world's first on-glass language model demo. So with that, I'll actually talk about how we have these three different models that are running.

Interestingly, I can get a language model to run on this frame, but I can't keep my mic on. But nonetheless, world's first small language model demo. So, we'll be talking about taking three models there. And if I can get the device, I'd like to give you guys a demo.

Okay. So, hopefully you'll be able to see what is being projected and what I'm seeing on my glasses over here. Ask your question after the chime. What is Newton's third law of motion?

What are the ingredients of fettuccine Alfredo?

Tell me a poem about augmented reality.

All right, so you guys get the idea, right? This is truly amazing. On this device, a light form factor device, we can run a full language model. This is truly, I think, really groundbreaking. And you can envision, right, the questions that I asked, they have to do with education. They're questions that have to do with, you know, let's say you do a quick run to a grocery store. What are the things that you should have on there? Even a poem. I know sometimes I have a feeling to, you know, hear a poem about augmented reality. Well, this device can do that for me, right? But I think if you notice, I'll hand this back to Cam over here. Cam.

All right. So coming back to this, if you notice, there were three different models running on the device, by the way, at the same time. There was a model that was actually converting my speech into text, which is about a 50-million-parameter model, an automatic speech recognition model. There was a second model that was actually the full language model, about a billion-parameter model, and then lastly a text-to-speech model, which is about a 50-million-parameter model. All of that happening on a super light device. And by the way, if you noticed, I was actually controlling the device with a ring. So, I'm also today announcing a smart ring controller for AI glasses. This has full gesture control. It has health monitoring. It has six degrees of freedom control built into it to be able to control a device that does not have a mouse. It does not have a pointer. It has nothing for you to be able to control it. So, super excited about it. But please take a look as to what the device can do.

Truly amazing. Right. So now, thank you. With this, you can actually, you know, bring in this experience. This is what we've done with our partner Kiwware to be able to really bring these experiences to a smart glass-like form factor. You can actually check it out in our booth as well to really see what you can do with this sort of a device. And now basically what we have done is we have a very, very complete roadmap both on the MR side of products, which is our XR product line, XR2 Plus Gen 2, and now with the inclusion of AR1 Plus Gen 1, we basically have the AR part of the roadmap completed as well. There's a lot more that's happening further to the right of this chart that we'll be sharing in subsequent AWE forums as well. But with all the announcements today, here is a quick summary. Very excited about the sports alliance. We want to bring our Snapdragon Spaces partners to Android XR, the ring controller, and the best-in-class AR1 Plus Gen 1 platform, which I think is the world's best platform, can do a 1-billion-parameter model natively, and I think that is something that truly excites me. We have a few other sessions coming with Say and with Simon. Please make sure you attend them. And these are all the featured partners that we have in the Snapdragon booth. I want to wrap up by saying, look, all the pieces that I've shown you today, this is how we're building this ecosystem together, but I need help from all of you in this room to be able to create these amazing experiences on these devices because truly XR is here, but the combination of XR and AI is absolutely synergistic, and this is something that's going to drive us to the next level and it's going to really increase the pace of innovation. Look forward to all the amazing things. Thank you.