In every era, there is a need, a spark, an instinct that brings us together to imagine and to innovate. This hasn't changed. What's possible has. We create technology that makes our world better. Ideas that move us, empower, and connect us. But great ideas cannot thrive alone. They need to be shared, challenged, refined. Through our insight and collective curiosity, we advance and grow. New solutions, new directions, new answers. We do that here together. This is the age of connected intelligence. Welcome to the IQ era.
Good morning everyone and welcome to day two of MWC. We've got a fantastic panel to kick us off this morning talking about what is the X factor in the next reality. So I'm really excited to introduce our fantastic panel.
Lovely. Welcome everyone. So, we're just going to do some quick introductions and then we'll get into the questions for our panel. So, I'm Lisa Kelvy. I'm the global head of infrastructure and transportation at KPMG. I'm personally really excited by this topic because the potential for the industries I work with is absolutely enormous and we have a fantastic panel here with the real experts this morning.
Good morning everyone. I'm Rafa Camargo. I work at Reality Labs at Meta and I lead a lot of the engineering work to develop our wearable products.
I'm Rob Morgan. I'm a storyteller and an experienced designer. I wrote the first manual for storytelling in augmented reality. I started life at PlayStation where I was a lead game writer working on augmented reality and virtual reality games. And now I work across museums and games and theme parks integrating augmented reality technology.
My name is Ziad Asghar. I lead the XR variables and personal AI business at Qualcomm. Really fortunate. I worked a long time on the smartphone side and now as we see the transition from smartphone into these variable intelligence devices or personal AI devices, I'm glad to be working with many of our partners on these technologies as well. Super interesting times.
Lovely. Thank you. So, we've got in safe hands this morning. We've got a fantastic panel for this topic. So, we're going to kick off with a question about the current landscape for XR. It feels like we're at a real tipping point now. As you just said, moving from the smartphones into other devices. Really quite exciting blend of human intelligence and then the extended reality. Let's start at this end. What excites you most about where we are in the next few months? What's coming up that's really exciting?
Yeah, as you said, I think we're like in a very interesting point and an inflection point. If you look back at the history of personal computing, you know, we've been giving ourselves more flexibility, more economics, more mobility to it. Mobile has been a big leap forward on that. But the one thing we haven't been able to disconnect from is like we are tied to screens, you know. So you're either on your virtual world or you're in the real world. I think with XR that's what we are changing. Suddenly digital and physical come together which is a lot more natural for us. You cannot say like oh leave aside your digital world, leave aside your physical world. They really need to come together. So we are at the point where finally the technology is maturing on that and we're not talking about like even next year the following year I'm wearing these glasses and they already have AI that is contextual you know I have audio and all that. Really augmenting my senses already and I don't need to be basically heads down onto my screen to do a lot of things as an example. So that's really the exciting part here is really the logical natural evolution to what is basically personal computing.
Fantastic. Rob, I know that you're really passionate about the content and the experience. What's exciting you?
That's the thing like as an artist and a content person right now we're also going through this transition from pass through virtual reality and pass through mixed reality into proper see through which has been the dream for so long that people like me have been talking about it for well over a decade and I think what's exciting for me is being able to put dimensions of adventure and gameplay into people's normal lives into people's ordinary walk to the shops because I think it's telling that still the most popular and successful and recognizable augmented reality XR experience is Pokémon Go. Not because it necessarily augments really sophisticated graphics into space, but because it adds this dimension of adventure and objective into people's lives. I'm really excited as we move away from pass through and into see through to be able to put experiences out onto the street that really add something to people's everyday and to create that kind of context of content in their lives.
Look, I think this is an amazing time right because like I said in the beginning now you're getting to these devices that the way I like to put it is these devices give you some sort of a superpower. It should allow you to do something that you cannot do today. It should allow you to be able to converse with a person that doesn't speak the language that you speak. It should allow a student to be able to understand where they're stuck on a homework. You know, it should do something for you that a smartphone today cannot do. And by the way, these devices are doing exactly that. I mean we have customers and we have partners who are basically using these AR-like devices to be able to actually you know allow the people who may be visually impaired to be able to understand what's in front of them. Right? These are amazing use cases that only a device like a smart glass or a device like a VR device can enable that a smartphone cannot do today. And the second part I would say that's really exciting is this inclusion and the merging of XR and AI. I think it's solving some amazing problems for the space. I really think these two are super symbiotic, right? Think about MR. What's been the challenge? Creating content. Well, guess what? AI is awesome at creating content, right? This is solving that problem. We can do models like, you know, text to 3D. We can do Gaussian splats to be able to convert 2D to 3D. These are exactly the challenges that we have had. And then on the smart glasses side, AI is the UI now that sits on this device. It's actually driving this amazing innovation into all these new form factors that are going to bring AI to the consumer while giving eyes and ears to AI. So super exciting times whichever way we think about it with AI plus XR.
Fantastic. And for me personally, you know, my background is infrastructure and construction. So my whole life has been about taking 2D things and making them 3D in reality. And I can see enormous potential for these technologies in terms of how we do that much more effectively. I think picking up on some of those themes there. I know Meta talks a lot about super intelligence. Could you bring that to life a bit with some sort of real world case studies of what that means for you?
Yeah, I mean Mark and Meta, we've been talking about super intelligence. It's a way to kind of like maybe express the AI in a more clear way to consumers. And what we mean by that is essentially it's an AI that is contextual aware that is always on with you and then it anticipates your needs and reacts to those and proactively works on those versus just reactively working on those. You're seeing just a beginning of that at this point. That's starting right now mostly reactive. You're still like querying it and all that, but very quickly you're going to start seeing things where it's like proactive and basically being one step ahead of you, always helping you because it really understands your environment. We're talking about like you cannot do that with your phone. Your phone is in your pocket. It doesn't know what's around you. It doesn't understand what you need in this moment. You know, you may get a message but it may not be the right thing to bring it up to you right now because you're in a very important meeting as an example and so on. So all these things are going to change significantly the utility of the product. And it's not just for consumers. You can also look at enterprise. There are a lot of use cases that we're seeing emerging as well on this. For example, there is a lot of field operators you know that need a lot of skills to like is there a repair or an installation and all that we're seeing those emerge. Suddenly you cannot have like your top tier support everywhere. So there are two parts to that. Like one you're like there dealing with something you're hands-free but AI is with you they're seeing what you see it can whisper to your ear it can guide you through that when with the AR display it can also literally digitally guide you overlaying a digital components to the physical things and then even when the AI is not ready for that you already are connected to maybe a tier three support that essentially now can scale because they can support in real time someone that is basically gone beyond the complexity of what they need normally do in the field. Yeah. So that's one example. We're also seeing other examples like for example in training you know they basically you're wearing this you're training on a new activity and all that they is really both guiding you but also seeing what you're doing where you're stumbling where you're doing it well and really coaching you through it. That will be another good example where you're going to see also in the enterprise doing these things. Same thing you can translate to consumers. Yeah. So, we're really really excited about this. We're literally at the tipping point of this thing and like actually the examples I just provided they're real. They're happening today and then you can imagine now extending this especially as the AR displays come on to be more accessible on these products as well which is really coming now together. We already launched last year first display glasses in market that are truly wearable but very light not like things you saw before that were honestly from a form factor were not there even if the technology already provided some of those experiences but the form factor is also fundamental so you're also going to see that in the consumers so we're really really excited about it.
And I can really see that because I know in the sectors that I work in one of the biggest challenges is time to competence so typically engineers field staff to be fully competent can take seven eight years and actually if we can use this technology to really get that down to enable them to do more and be supported I think there's a huge productivity potential absolutely with that. So coming on to you Rob in terms of the experience I know you've mentioned museums education what do you think are the things that are exciting there?
I mean I think that AI as Ziad was saying like AI is a force multiplier for what we were already trying to do on your smartphone in terms of encountering the real world. That's what's really exciting about it, being able to seamlessly give you like information that's fed into your field of view as you're encountering like real things rather than things just mediated through a screen. So from a game perspective, AI is not just improving the pipeline of being able to make stuff, but also it's unlocking the capability that's easy to imagine, but which has been so impractical for so long of reskinning a real environment to make it look like a Victorian fantasy or make it look like you're living in a Nintendo world. Like that kind of game and ambient fictional experience is really exciting to me as a storyteller. But then if you think about a cultural experience like visiting a museum, we've been looking for an interface that allows you to kind of curate your journey around a museum for many many years. And like apps can only get you partway there because ultimately you don't want to be looking down at a screen. You want to be out and in the environment. And so it's easy now to think about being able to curate something that is designed for you in terms of a route or even an explanation or a curation of a single artwork or artifact. But I also do want to kind of I think we need to be conscious as developers and creators that ultimately going out in the real world is an encounter with the unexpected and the unusual and things that we weren't necessarily ready for. And that's a good thing because that helps us grow. I think that at times we've verged on kind of the danger of only seeing what we expect or what is being curated for us on the internet. And I think that causes people's minds to close somewhat. So I'm also very conscious that as we're creating these kinds of interfaces, it's really important that we maintain that encounter with the unexpected that the real world represents.
Yeah. And I think that's a really fascinating point, isn't it? And I certainly could do with the AI guidance for this event here. It's something this enormous finding your way around. You can see that being really powerful. So we'll come back to the topic of the unexpected and how do you sort of avoid getting into a bubble but let's hear first what are you excited about?
I think some of the points that Rafa mentioned I think from the super intelligence perspective are very interesting but maybe taking a step back right we've seen these evolutions right we had the PC and the computing stage and we saw the evolution to mobile and then we saw the evolution now to these new kinds of devices and there's always been a new human machine interface that's enabled it so if you think about it was the mouse and the keyboard that enabled the PC it was really the touch screen that enabled the mobile but now for these new devices that we wear around our body like glasses or you know pendants or necklaces or what have you. These the interface essentially is AI. So what we have done at Qualcomm is that we have actually added in a lot of the capability on our products. Last year at one of the conferences I actually showed a small language model running just on the smart glass. So this is a scenario where you know just like you have a chat GPT session but now you can have that session with your glass and that information is not going to the cloud it's not going to the phone it's all being able to be done just on the glass that's kind of the power of how much AI we have in these devices but the really interesting thing about these devices is when you do this session on chat GPT let's say or a different AI you answer the question or you ask a question or you know your 10-year-old asks the question you get back the same response there is no personalization of that AI experience. I think what we envision is that with the fact that now there's a camera over here and you can actually get that multimodal input, you can actually understand the context better. Now you start to personalize that AI experience. So it's not that generic answer that you give to everybody. Now you do an answer for Rafa, you do an answer for Rob, you do a different answer for me. Depending on the history and depending on the context and there's different context that these devices provide. The glass tells you what you're looking at. The smartwatch will tell you, for example, your heart rate. It will tell you your activity level. It tells you are you walking or you're running. You know, your phone tells you of your location to the nth degree. Just aggregate all of that and add that into your AI and you have an experience that's far more curated and personalized than ever possible. That's kind of what I'm looking forward to that I think we'll be able to bring in coming days with these devices.
Yeah. Yeah, I can see that. And you mentioned in your introduction about the engineering of these things and you know the real challenge of getting wearables that are also things that people want to wear. They've got to fit with comfort and all of those things. They've got to fit with accessibility. They've got to fit with fashion and looks and the image you want to present to the world. What do you see as the biggest challenges in getting wearables that people really want to wear?
Oh, lots of them. First I think so far we found actually the glasses form factor is the most ubiquitous from with regards to enabling these kind of experiences. A simple reason you know literally glasses are close to your senses your more important senses your vision your hearing they can talk to you they can see the same thing that you see they can hear what you're hearing in exactly the same egocentric way that you do it. There'll be other form factors that will emerge and all that Ziad was mentioning a bunch of them right now but this is the one we have invested the most because it's the most natural one but it's also one of the most challenging one first our face is our identity I don't know how many neurons in our brains are just dedicated to basically distinct people I cannot remember people names I remember everyone's faces isn't it and nature really designed us for that and which means it's very different than the current things that we see in consumer electronics or your personal computing even a smartphone probably put it out here we all have the same one or they all look the same suddenly you know you want to choose style and I cannot do like three styles for all the world like even the portfolio we have out today and it's the beginning we had like I don't know like over 50 different SKUs you know and it feels short. So that's already a challenge. It's just a style. So you get acceptability because essentially you really really want to personalize it. The second one is you were mentioning is ergonomics. It is essentially you want to wear this all day. So it's useful to you. It's always with you as an assistant. It really need to feel light and it needs to feel like you're wearing it all the time. So and the third thing you were mentioning it at the same time it needs to be very powerful from a compute perspective. Indeed. So it can really do this contextual understanding. It can be basically even when we bring the AR displays you know we have literally single digit count with one hand milliseconds to do what we call photon to photon. So like we see what's going on in the world. I want to put something digital on top of the world and I have four five milliseconds to draw everything and place it there. Which means it needs to happen here. I don't even can go wireless somewhere else. I don't have that amount of time. So when you combine all these things, they're going to be fighting the ergonomics. You want long battery life. You want to use it every day. That creates weight. Batteries are not improving at Moore's law or the way silicon is improving. Even silicon is not improving now fast enough for the amount of compute we want to put here. And so that's like a significant dimension just like on the device itself. And then we've been talking about the AI and that's another challenge. AI needs to become truly multimodal. You mentioned a bit that before it really needs to be models that are basically native multimodal video, audio, speech and it's not like just what you're saying, how you're saying it and all that. So it feels natural. It feels really contextual aware including to your intonation, isn't it? And then the other part, it needs to be super trustable. We talked about that. And in addition, it needs to find the right balance. It's not a good assistant that is nagging on you because now it's capable on you all the time. Don't do this, don't go there, like do that. It really that contextual awareness that we talked about needs to be very sophisticated. It needs to be at the right time that is interrupting you or proactively going to you at the right time receding and saying like I don't need to do anything for you right now. You're taking care of it. Those are the challenges and they're going to be for quite a while. We're making a lot of progress in form factor. We already launched our second generation. It's much better. You'll see new generations from us launching in glasses both styles and comfort. But the AI is probably the one that well it's evolving super fast because actually we're already doing things that we thought like we'll be doing two years from now not today. So I do think that also is going to converge quickly but all these things need to come together to have a mass adoption and then the last piece is going to be essentially price which I think as it grows we have economies of scale so it's accessible to everyone.
Yeah. So coming to you Rob and maybe this is a bit of an unfair question, but you've talked a lot about creativity, a lot about storytelling. You know, I think we're quite fixed on things we know. We're fixed on watches, we're fixed on glasses. Is there something else we need to be inventing for wearables?
I think it's worth talking now about interoperability because for good reason. It's kind of an old-fashioned buzz word, but the spatial internet, which I think is still a useful concept, that's not going to be built necessarily on TCP IP and the same kind of inherently interoperable protocols that the original internet was built on because it's being built out of proprietary hardware. It's being built out of proprietary data sets and 3D meshes of the world. And a lot of it is going to run on proprietary AIs that have been trained on proprietary data sets about the world. Like there's a data race on now to get that data to train those geospatial AIs to be really good at being able to interface with the world to be most useful to users. But equally, I think we do need to think about interoperability because if you look at again kind of an old-fashioned buzzword in some ways, all of the metaverses that have been successful and been compelling for users, things like Roblox, things like Fortnite, places where people, particularly young users, choose to dwell and spend their time, those are built on platform agnostic interoperability. And so I think when we're thinking about when I think about the kind of creative experiences that I want to be building in the glasses, if I want to sell or I want to provide a partner the opportunity to sell a cool skin to somebody in augmented reality so they can walk down the street with like, you know, golden flashes going off around them or wearing like cool augmented reality Nikes over their real shoes or be part of a game world in that way. You sell that kind of product, the user needs to know that a decent proportion of other people are going to see it. And so I think as we're developing these ecosystems, we also need to be thinking about the kinds of experiences that are going to run across hardware in these ecosystems and developing towards that so that we can create experiences that are competitive and compelling for people and that you know allow people to dwell within an augmented version of reality that isn't really kind of closed down and private and sort of encrypted.
Yeah. Brilliant. Ziad, for you, where are the biggest challenges with wearables? Maybe the biggest opportunities.
Yeah, I think Rafa covered very well on the device side. I'm going to go more to the actual silicon product in a way. And I think to think about it, one of the biggest challenges is power. The interesting part is if you really look at your mobile, your typical mobile has about a 5,000 mAh battery. A typical smart glass has a 250 mAh battery. That's a twentieth. But if you go and talk to the consumers, they want the same quality of the image, they want the same experience that they have on the smartphone on the glass. That's really the challenge. So that's really what we are very focused on at Qualcomm of really taking and miniaturizing these such that you're actually able to give that same experience in a very small form factor at a fraction of the power consumption. Now this is a very interesting problem. It's a very Qualcomm problem because we've always lived on devices that are not plugged in. We've always lived on devices with very small batteries. So, we are extending that learning that pedigree further onto the side of being able to optimize the power consumption even more. We're taking a three-pronged approach. So, at the chip level, what we do is number one, we're going to move to a much more advanced transistor. As you go to more advanced transistor, better nodes, you're able to get some benefit from there. Number two, what we do is we take very unique architectures. So these architectures are different than what you would do on a mobile product or what you would do for example on a PC product. These architectures are where for example you do a lot of processing just on the chip. You don't move off to the memory that you have outside because in terms of moving data you lose a lot of the power consumption. And then the third aspect that we do really is being able to actually take each and every one of the functions like the AI block inside like the camera block inside like the video block inside the modem connectivity. We basically reduce the power consumption of each of those. So we're taking that very meticulous approach to be able to actually reduce the power consumption for these products. That's one I guess big set of problems and challenges.
I think the second one is really about experience, and I feel this is what is absolutely critical. The power of having an amazing reasoning model where you can basically have a long task and you can actually break it down and the device basically does that task for you is amazing. But on the other hand, if the response that you give your agent, the response that the agent gives is not right, people will not use that function again. So the bar is really, really very high to be able to have a pristine experience such that people start to rely on a device like this, because otherwise they're like, you know, the experience wasn't what I wanted. In the same way, I mean, I was talking to some of the customers in China and they were telling me that I want exactly the same camera quality that you have on the highest-end smartphone. Well, the space is much more limited, but those are exactly the kind of challenges. We love these challenges and that's what we're working on doing. So, I think experience, experience, experience and power, and I think those are the things that we need to solve as an industry to be able to give those right experiences to folks.
Yeah. And just, you know, open to any of you really who wants to come in on this point, what about the connectivity point? Because obviously the quantities of data that we're going to need to be transferring to and from the devices is going to be enormous. And you know, frankly, last night in my hotel, I couldn't even get 5G to work. So, how big a roadblock is that?
I'd love to comment on that. I think with this new paradigm shift that's happening with these personal AI devices, I like to call it that you need different roads, and these different roads are things like 6G. So like one of the interesting parts is for example, if you're doing multimodal AI, you will need to take a picture and send that either to a device, do it on the device, in certain cases send to the cloud. Typically the networks are designed such that you have very good downlink speed but you don't have as good uplink speeds. Some of the great things that 6G is doing is actually they're going to augment the capability of having some degree of symmetry in downlink and uplink rates. I think that's going to start to solve a lot of challenges. Number one. Number two, the bits per second per watt as to how much power you spend to be able to send down data or send it upwards is going to improve also with 6G. But the third thing is really to have this proper hybrid-like scenario where you do some processing on the glass, let's say, some processing on the phone, some processing in the cloud. It requires you to have this seamless connectivity and that means that even if you are at the cell edge, you still want to have very good connectivity. It's not that you are in the center of the cell, you have great connectivity, you go a little bit further and you don't have that level. So those are exactly the challenges that 6G is going to be addressing. And I think we have done of course a lot of innovation also on the side of personal area networks like Wi-Fi and Bluetooth to be able to optimize the power further. We need to do all of these things to really make these devices successful. So here at Mobile World Congress, we actually launched our new platform that we are calling basically the Snapdragon Elite platform and that platform basically has a hexa connectivity just to give you an idea. It has 5G RedCap. It has Wi-Fi. It has Bluetooth. It has GNSS capability. It has UWB. And it has non-terrestrial network satellite as well. So again, you need hyper connectivity to really be able to give the full capability of these devices.
Yeah. And again, I can imagine that then conflicts with the weight and the wearability and the battery. So, uh,
