Hi everyone. Welcome to Oriel College. My name is Ben Caldecott. I direct something called the Oxford Sustainable Finance Group based at a bit of the university called the Smith School of Enterprise and the Environment. And we're hosting this lecture today at Oriel in collaboration with the Oriel Environment Group because it's the 700th year anniversary of the college, and as part of the kind of celebrations, a variety of lectures being held here on the environment, and so that's part of this series.
And it's a great pleasure to have Will Marshall here, who's going to give us this lecture. And Will is the co-founder and CEO of a company called Planet, a publicly listed company based in San Francisco, and they operate the world's largest constellation of Earth observation satellites imaging the entire land surface of the planet every single day at a high resolution and different spectral bands. And you're going to hear more about those capabilities shortly.
Now, Will is a physicist by training, did his doctorate here at Oxford, so it's great to welcome him back here. Now, why does this matter? And why is it a great pleasure to host Will at Oxford? So, in our research group focused on finance and sustainability, we have something called the Spatial Finance Initiative which is working to bring geospatial data and analysis into financial theory and practice. So, we rely on the data that Planet and other Earth observation companies and providers create and disseminate.
But, we're not the only people in the university, of course, that draw on these capabilities to understand what's going on in the world and where it might be going. So, some examples. So, colleagues in archaeology use this imagery and this analysis to find archaeological sites, to monitor damage to heritage in conflict zones, for example. We have colleagues working in international relations and security, researchers tracking what's going on in conflict areas, and obviously what we see day-to-day in the news in Ukraine, for example. A lot of that is satellite imagery, a lot of that's coming from Planet.
There are also people working on humanitarian questions, mapping refugee camps as they grow and assessing damage after floods and earthquakes. We have economists and development researchers applying this to development economics, measuring urban growth, mapping poverty. And we also have colleagues in natural sciences, including Yadvinder Malhi, who's here at Oriel, who are monitoring deforestation, land use change, biodiversity, tracking floods, water scarcity, and also methane leaks from facilities. So, a whole range of different use cases, and that's just scratching the surface.
The availability of current and historical Earth observation data at the quality and scale that's now possible because of Planet and other providers is revolutionizing whole fields of study and opening up new ones. So, being up to speed with all of these developments means a great deal to us as scholars and researchers, which is one of the reasons why we're obviously very happy to have Will today.
The other thing I wanted to briefly mention, obviously Will and Planet also sit at the intersection of two really important economic, financial, and technological forces at the moment and into the future, and those are of course space and artificial intelligence. Today is a big day if you're interested in that intersection. If you're interested in seeing those forces at play in financial markets with the SpaceX listing today, the world's largest IPO, the largest IPO in history.
And some of you in about 30 minutes, 23 minutes when markets open, might directly own a stake in SpaceX, and those of you who don't will basically automatically have exposure to that through passive funds in the next 2 to 3 weeks as those passive funds are forced to buy into that stock. So, there are literally trillions of dollars on the line in relation to space and AI. And Planet in a way is part of that interest.
So, I'm sort of hoping that we'll in addition to his lecture and the Q&A, there might be some questions related to all of that. So, it's very exciting stuff. Great to have you here. Will's going to speak for about 30, 35 minutes. We've then got time for Q&A. And then we've managed to organize a reception in the quad that you came through, a Pimm's reception. A bit early, but it is a Friday in summer, and I'm told it's not going to rain. So, hopefully you can stay around for that. Without further ado, over to you, Will. Thank you.
Thanks, everyone. Thanks, Ben. I also wanted to start a little bit with why it all matters. And I love this picture that was taken not by Apollo astronauts, but just the astronauts around the moon just a few weeks ago. And it was cool because just like the Apollo astronauts, they reminded us we live on this little fragile planet in the vastness of space, and it's very unique. And I actually sent two probes to the moon when I was at NASA. One that found water on the South Pole of the moon is very exciting. Makes it easier to potentially put people there more long term and stuff. But whilst I was doing that, my co-founder Robbie was working on a mission that was looking at exasolar planets. So, planets around nearby star systems. And it went on to find thousands of said planets. But I'm here to tell you unequivocally that this one's the best one.
So, we started Planet with a mission of helping us to take care of the Earth knowing that you can't manage what you don't measure. And in particular, measuring things on a human time scale means measuring them on a human time scale. And so, we took it upon ourselves to set the goal of imaging the entire world every day.
Yeah, that was where we were going. Oh, I see. You have to sit there, do you? You have to. So, just stand there. You're right spot. Image the whole world. Yeah, it looks like it needs line of sight. Okay. And make it accessible to everyone. And I'm going to be talking a little bit about that today. And it has a lot of different applications because of digital transformation, sustainability, and peace and security.
Okay. And what's underlying this, just a tiny bit of background on space, is we've seen rocket costs come down. You've heard about that with SpaceX. We've seen about the four to five times cost reduction in launch cost. But, the less known fact is how much improvement we've seen in cost capability of satellites. Over the same period about the last 10 years, we've seen about a thousand times improvement in the capability you can put in each kilogram that goes on that launch fairing. And that's really what's underpinning the proliferation of satellites that we see now in space.
And all of it leads to more data. And more data either being transported around the planet with things like Starlink or OneWeb or being taken off the planet. So, the upshot is a data revolution. We spent a lot of years trying to miniaturize our satellites. And then we launched them on lots of rockets. And this one was just a couple months ago from Vandenberg Air Force Base taking some of our latest satellites into space. It's cool to be able to go and see those. And we took some of our latest customers to see it as well.
And this little video shows the satellites popped out. That's one of our satellites. They popped out in space. It's almost like a bus ticket these days and it's very different from when I started my career in space. So, and then what they do all together, hopefully this will play, is they act like a line scanner for the planet. So, they're all in a polar orbit. Each one takes a strip of images as it goes around, but the earth rotates underneath. So, the next one comes along takes a strip slightly next to the prior one such that over 24 hours the earth does the legwork during the rotation. And the satellites stay in a sun-synchronous orbit.
We have a bunch of other satellites that image in different areas. So, this is our complete fleet. So, we do have a daily scan satellite of Doves. SkySats are higher resolution systems at 50 centimeters. Tanagers are our first hyperspectral satellite. We're launching a couple more of those this year. They can track permissions. And Pelicans are our new high resolution satellite fleet. And it gives us gobs of data. So much data that most people can't use it, which is why I'm going to talk about AI and how that's going to help make it easier.
And we serve that all up on a platform. And it has various applications and I'll just talk you through a few very quickly because I want to get on to where AI and satellite data are going, but we do a fair bit of work in agriculture helping farmers around the world improve crop yields across millions of farms. We can tell the crop type and crop yield in each 3-meter box of the farm field. We just announced a partnership last week, for example, with John Deere. They're using it for precision farming of their tractors.
We do a fair bit of work in fire prevention by monitoring people's electrical power lines across California, for example, to look for encroachment of trees onto power lines. We do things like, I'm not sure if I agree with this, but stopping cannabis growing in Humboldt County. This is legal, but they have to have a permit and they don't often get their permits because they're lazy and we catch them. And drought insurance. So, instead of having people going out to the field, AXA and Swiss Re actually just do it automatically from their laptops.
We do a fair bit of stuff in disaster response. So, this is the Lahaina fire. FEMA used this. The American Red Cross used it to help save lives. We do a lot of work with Ukraine. We're the only ones that monitor all of Ukraine every day, all of Russia every day, helping them track security threats coming their way. Including this, which was the first bridge that Mr. Putin and his team built across a river between Belarus and Ukraine. The day when he said, 'No, we're not going to invade.' And then we were like, 'Well, what's this then?' There's a bridge to bring troops across into Ukraine.
And ever since we've been helping them building damages assessment for the UN Development Program and have an active standing here for formerly at the UN monitoring the health of all the crops across all of Ukraine for the World Food Program and things like that. And shedding light on this for everyone to see, trying to get at the truth about what's going on the planet.
We do a fair bit of work in sustainability which was obviously relevant here today. Through these what we call digital public goods projects. So, in this case, we've mapped all the world's coral reef systems and give alerts of bleaching events and fishing vessels that shouldn't be in marine protected areas. We help another UN program track deforestation across 64 countries, UN FAO. And this little file here that's probably not going to show is showing some of our work in tracking deforestation across the Brazilian Amazon.
And this is the upshot statistics that over the last 3 years, so Brazil Federal Police, we find illegal logging roads, illegal narcotics, illegal mines, and then they actually send people up to these places where we alert them about. They do that about a dozen times per day, and they've confiscated now $3 billion worth of illegal mining and other narcotics and deforestation equipment. Sometimes they send us the imagery, they blow up these machines because they can't confiscate them. They just put TNT on it. But most importantly, we've helped them to reduce deforestation rates about 60%.
This is our first work in global carbon. So, this is a map of all the carbon at the individual tree level across the planet, and we produce a quarterly map of every tree on the planet, how much carbon is in that to underpin and hopefully help carbon markets, and that's partly why we're here.
And the Tanager spacecraft can monitor emissions. So, it's a hyperspectral satellite. Methane sticks out like a sore thumb in a few of the very specific spectral bands. So, this mission looks along oil and gas pipelines, fire and waste facilities, and other things. Finds emissions, and then helps people, operators, to turn them off. Everyone doesn't want an that. So, between those two, I think that we can help monitor the carbon above ground in the trees, the carbon emissions from every point on the land over the earth, and hopefully blue carbon and brown carbon, the soil carbon in time to have a quantified metric for every place on the planet.
And the final layer of applications actually in academia, where a lot of people use our data to track system sciences across all the different kinds, atmospheric science, cryosphere, glaciology, all the different pieces, climate work. And most of that's funded by NASA that provides this data for free for all universities across the United States. There's a somewhat similar program with the European Space Agency you can apply for if you're interested, but Oxford has a program so you can have access to all of our data, too.
I want to talk about AI mostly for the rest of the discussions because AI and space are getting married. We were doing a fair bit of work with CNNs, the old school way of tracking objects, but now we're doing a fair bit of work with LLMs, and it's fantastic what they can do. A lot of this based on the fact that we have 3,000 layers of images, so 3,000 layers of images for every point in the land mass of the earth on average, tracking changes over time. Obviously, AI's only as good as the data it's trained on. And the AI algorithms today are primarily trained on the text of the internet. And they're incredibly fluid about understanding a lot of different pieces of human knowledge that's captured in that text on the internet, but they didn't know much about the real world.
So, if a farmer asks, 'How is the crop yield doing in their field?' it can't tell them very much about it. It can tell them about the theory of agronomy, but it doesn't know anything about their field today. So, this is some of the work in CNNs. So, this is like, 'Can you see the ships and the planes?' The AI is very good at pulling that out.
And in 2018, I did a TED Talk called Queryable Earth, where I was basically postulating that if we could see every ship, every train, every road, every building around the whole world every day, then we could index everything on the earth. So, a bit like Google indexed the internet to make it searchable, we're trying to index the earth to make it searchable. 2018, fast forward, LLMs have made this goal way, way more tractable and faster. And so, I call the combination of Earth observation and LLMs planetary intelligence, because I think it's a new era of intelligence.
But a lot of people are talking, Dario from Anthropic and Demis from DeepMind and others are talking about building physical world models. And obviously for physical world models, you need real-world data. And I think what that enables you to get is sort of a planetary intelligence. Obviously, the natural place for planetary sensing is done in space. And soon we'll actually see some of the compute going to space as well. And I think this is a new era.
So again, LLMs know about floods, but they don't know about the flood in your town today and who needs help to rescue that. And that's what we're trying to change. So I want to actually show you this little application we're building, which actually merges LLMs with our data. It's in private beta now, so it's an early test app. But I'm pleased to show a little bit about what it does. So basically, it enables you to search the Earth. It would do text queries in natural language. It can search the whole planet in about a second for any object that you either find and say, 'Find me more like this.' Or ask questions about.
You can also just give it like, 'Find me blue ships in lakes near bridges.' And it will go and find those things in the whole world. It will write descriptions about it. Come up with before and after certain events. Do analysis. And we've just added this new deep research mode, so you can have it go off, do analysis over larger areas that are more systematic. So it's kind of fun.
This is some little description. You can even write code. So it can build applications for you in there so that you could build a dashboard or build an application in time. So this is some analysis and if you go to the next slide, it can generate graphs and reports. If you go to the next slide, maybe I should just stop trying to click this thing. Yeah, push that video.
This is a real one that it did where we were asking about a flood plain and assessing the flood plain over time. It did all that assessment itself. And drawing on the world's knowledge that it has latent in the LLM together with the data that it can access from our daily scan for the last 10 years. And in this case it is spitting out all these anomalies, gives you quantified analysis. It may not be perfect, but anyone can have access to it. And that's the beauty of this system.
And I really hope that this will open up geospatial data to a lot of people. You know, it's basically the big organizations, the big ad companies, the big countries and so on that can get access to satellite data today. But I hope this application will start helping us to have a wider range of applications. And most importantly, to take care of course our precious planet and biodiversity on it. It's really useful for conservation. And I had a friend the other day who already found some encroachment on some parks in Africa using this.
And this is where I wanted to talk a tiny bit about where this might be going next. So click on that video again. So basically we're putting Jetson Orin platforms on our satellites so they can do processing at the edge. And the whole idea, this was the first example of this. We did this just a few weeks ago where the satellite took the picture of this airfield in this case in Australia in Alice Springs and automatically did plane detection within a minute and sent down just the answer to the question right there. So instead of having to wait a few hours for it to go over our ground stations, it can now do the processing at the edge and then satellite-to-satellite communication so that you get the answer back real time. Obviously in cases like that Lahaina fire or other disaster response, getting answers in minutes could be really, really helpful.
And so I said the natural place for sensing the planet is in space. Well, soon the natural place for putting compute is going to be in space, too. And this sounds super sci-fi, but is actually coming to us to be reality very, very quickly. So basically, we're working with Google on an early project moonshot, as they call it, called Suncatcher, which is flying their TPUs in space not for processing a sensor that's already there, but actually just to do compute in space for its own sake. Why? Because it's actually just going to be cheaper to put it in space in the next few years. With lowering launch costs, data centers are primarily an energy challenge, and you can get five times more power for each solar panel in space. You don't have to have your buildings, you don't have to encroach that land, you don't have to raise up the energy costs of the local community or use their water.
You actually just put it in space. So we did an analysis about eight or nine years ago with Google that showed that when launch costs come down to about two or three hundred dollars a kilogram, it actually is just cheaper to put it in space than on the ground. It also has, I think, sustainability advantages of not conflicting with a lot of stuff on the ground. This is not a picture of the real satellite. I can't show you one. So this is a silly mock-up. But what might that enable? So that's the next phase in planetary intelligence when we put the sensing system and the compute systems together in space enabling us to have real-time understanding of everything that goes on in the planet.
And you know, just my hope is not just that this data helps us make smarter decisions day-to-day, but hopefully it will help us to level up and become more wise as a species as the de facto stewards of this planet. And so maybe I'll just wrap with a just a bit of a speculation.
I recently wrote a little article on planetary intelligence and I raised the specter of whether as we give AI, which is largely this disembodied brain in a vat right now, access to sensing data, will it therefore not just by understanding the world, will it come to value the world more, the physical, all the beautiful biodiversity on the planet? And, you know, I've recently got into birding because my partner's into birding and she's obsessed with it, so I go around. And the more I understand birds, the more I value them. You know, it's not just a bird over there, or it's these kinds, and I know there's tweets and stuff.
And so if the AI understands all the river deltas and the forests and the beautiful ecosystems of the earth, will it inherently value them more? I hope so. This is not, I think, a way out of the alignment problem, but I think it's an important component potentially of us aligning AI. Because the biggest threat we have to ourselves now, we just sort of muddled through the nuclear age, is going to be how we take care of giving birth to superintelligence in a way that continues to have the beautiful life that we have on spaceship Earth. And it's a tantalizing prospect that, you know, life is rare in the universe and there is a worry that one of the reasons, one of the genuine answers to the Fermi paradox, is that life gets to this sort of stage of technology and then blows itself up.
And so how we steward AI in particular is so important. And maybe, just maybe, if we give it more understanding of nature and biodiversity, maybe it will care about it. And I just do one final speculation. Here's me back when I was a student here with my supervisors. And this is Roger Penrose who I just caught up with 2 days ago. He's 94 now. And he has some, apart from his great work that he got the Nobel Prize in for physics for in black holes, he did some more speculative ideas about consciousness.
And I think that LLMs and those AI systems that we use today are going to have a hard time becoming conscious without sensory input. So, just like a baby is unlikely to become learn and become intelligent and become conscious without sensory input, how are LLMs going to become conscious without that? And he thinks they're never going to become conscious. I disagree with that. He thinks Turing machines can't do it, but I am a functionalist and I think that it is very likely that as it gets smarter, it will become conscious. And to all intents and purposes, if we are conscious, it's conscious. And so, anyway, as we, so, a further speculation is if it is not going to become conscious, it's not going to value, it's not going to value humans unless it becomes conscious and it needs sensory data in order to do that. And I think that will behoove us to help it to do that. So, we're trying to do that. Very speculative.
But much more practically, if you have any interest in playing with this app, it's in private beta. And you can literally just log on now and we'll give you access, and all we ask is that if you're interested, just play with applications and see what real-world value it gives you and tell us a little bit about that. Because I hope that if we spread this geospatial technology out to more people, we can have more impact and help us to take care of spaceship Earth. So with that, I'm done. Thanks very much, everyone.
So far, yes. They only get words. Yeah.
Yeah.
For sure. That, I mean, that's the whole point of what we're talking about here. Yeah, and I like to talk about the evolution to large Earth models instead of large language models, and that's the integration of physical world data. Obviously, satellite data is a core, not the only thing. If you think about LLMs, obviously they scrape most of the internet. They take in Common Crawl, which has taken Wikipedia and Reddit data feeds and all the rest of it and put that into their model. Your photos of cats and dogs online. But the most core thing I would say is there was a crux data set for those LLMs, it's probably something like Wikipedia, right? That's probably a lot of its valued data set.
What's the equivalent for real-world models? I would like to say, and I'm extraordinarily biased, but I like to say it's satellite data. What's the thing that has already basically the whole world monitored on a regular basis? The only thing that can do that systematically and comprehensively is satellite data. Now, there's lots of other things, IoT, soil samples, sampling, what have you, people's phones with various other things. They're going to be supplementary to a core corpus that can upgrade us from large language models to large Earth models. And I think that has a lot of implications like we've discussed.
In earth observation or in space journey?
I mean, I think we're going to continue to improve the data sets more. You know, there's three main axes you can improve: resolution, temporal resolution, and spatial and spectral resolution. And we're improving all three and many others are as well. So, our Pelican spacecraft is taking our 50 centimeter high res system and going to 30 centimeters as well. The hyperspectral imager is 400 bands and we're trying to improve that. And then the latency we're trying to improve as well.
Our next scan system is going to go from a 3-meter resolution system to 1-meter daily scan of the whole planet. And also come down much faster within half an hour rather than a few hours as it presently is. So, it's about a 10x improvement in the amount of data collected and a 10x improvement in the latency. So, we're going to collect better and better data, but I think the big unlock is frankly all the AI. You know, most of the value of our present earth imaging data sets again not just Planet, but all of them, the earth observation systems like Sentinel, like NASA's Landsat program, other commercial players, I would say less than 1% of the value is getting to the people that need it today. So, that's a tragedy and
That's more unlocked by the AI systems and analytics that make it easier so you don't have to have a PhD in remote sensing to get some value out of this. And that's going to happen really quickly, because it doesn't require obviously launching new satellites and stuff and a lot of the tech is actually there finally now.
