PSW 2520 Revolutionizing Earth Observation with Small Sats and AI | Will Marshall

Good evening everyone. So I'm looking forward to this discussion. I'm going to leave plenty of time for Q&A. Can everyone hear me? Okay, super. Okay, let's get in. So, space and AI. We wanted to talk about this and Larry asked me to start with a few remarks about my background and so I chose to focus on a couple of my inspirational mentors. And it starts with this guy, Patrick Moore. This is me when I was 17. I built this telescope on the left, and Patrick Moore is an eccentric astronomer, what he is at the very least, you can see with his monocle up here. So, Patrick ran this program called The Sky at Night in Britain. He is a sort of popular space program. It's a bit like Carl Sagan of England but much, much more eccentric. Anyway, he guided me a little bit. And then here's me doing my PhD, got a little bit more serious. But here my professors Dirk Bouwester and Roger Penrose were doing quantum optics and looking at studies to look at foundational issues between quantum mechanics and general relativity. And then my third mentor to speak about is Pete Worden. And he was the director of NASA Ames Research Center and a sort of maverick guy who oversaw that NASA center, having been an Air Force general but also a PhD in astrophysics. And he really shielded us, enabling us to do a little bit more out-of-the-box thinking.

As you'll hear about, and it starts with this one on this slide. So, the LCROSS mission was one of my first exciting missions that I was part of at NASA. And this was a mission to the Moon looking for water on the Moon. And excitingly, we found lots of water on the Moon, about five or 6% by mass in the area we impacted. So it was an impactor that slammed the south pole of the Moon in one of the permanently shadowed craters and the plume of material was then subsequently flown through. So we actually directed the upper stage of the rocket into the Moon and then this probe, the spacecraft you see here, flew through the plume before its own demise, but gave us a couple of gigabytes of data which took a couple of years of my life. But for us scientists, we like the front covers of Science magazine, and I was just sitting next to Marcia McNutt who was the head of AAS at that time. Anyway, one of the interesting features of this mission was that we did it in only a couple of years and it cost $79 million, which sounds like a heck of a lot of money to most people, but for NASA, this was bizarrely cheap. Because everyone at NASA headquarters told us that we couldn't do such missions for under a billion dollars. And we'll get into why a little bit. But it got us thinking about could we do spacecraft missions at radically lower cost than that. And so a colleague of mine at NASA had kept on holding up his phone when he was in our office and saying this has most of what a spacecraft has. Why are we making them so much more expensive? In particular, they have fast processors, they have radios, they have of course camera systems, they have accelerometers. If you look at the list of things a spacecraft has and you look at the list of things a smartphone has, it's probably 90% overlap. And so we took him seriously one day and threw a few into space to see if these things would work in space, and they worked just fine.

And these pictures you see here aren't very impressive pictures of the Earth. Obviously, more impressive pictures of the Earth have been taken. But they were taken from a phone camera that cost a few hundred dollars. And we were asking the question with this technology experiment: if this cost $500 and a normal spacecraft mission cost $500 million, what are the extra six zeros doing for us? And that is what led to this, which was we started Planet as a sort of spinout of NASA to experiment with much lower cost ways of doing spacecraft and leveraging some of the innovation that had been done by many others on faster, cheaper, better spacecraft missions like Surrey Satellite Technology and some of the US Air Force work in this. And classically, we were always told that you can only do two of those three things, faster and better and cheaper in space. Pick your two. But we were really trying to do all three. And so we left NASA to try to do that, and in particular with a goal stated here, which is to image the whole Earth every day and to make change visible, accessible, and actionable.

And the concept was very simple. We live on this beautiful spaceship called the Earth, all 8 billion of us astronauts and all the life on it, trillions of creatures. And how can we take care of it if we don't have data on the time scale of the change that we're making? You can't manage what you don't measure. If we have these challenges around the planet, and here's just three of them, there's obviously myriad more, you need data on the time scale of change that is happening to intervene. And actually, sometimes to my space friends I make the analogy that on a spacecraft, of course, if a spacecraft is spinning at say one hertz, you better not be measuring it once every hour if you want to stop that spin, right? And humans were operating this spaceship Earth with measurements of the Earth once every month or so from Landsat, but our operational cadence as humans and our interventions take place on days or hours sometimes, or weeks. But if we want to do something about that, and that is to take care of our spaceship, we better do something faster. So we set about this, which was essentially erecting a line scanner for the planet. The idea was to put up more than 100 spacecraft. We calculated we would need each one taking a strip map of the Earth's land as it went over in a polar orbit, a sun-synchronous polar orbit, and then collectively they would end up line scanning the planet. Actually, the Earth does the legwork in doing its nice rotation. The satellites actually stay fixed with respect to the sun in their angle. Then, sorry for the extra lines, but we then launch a bunch of others in all sorts of other orbits. But basically, we line scan the whole Earth every day at 3-meter resolution, eight spectral bands. And then we have a set of 15 satellites that can zoom in on any particular location up to 10 times per day at 50-centimeter resolution. So think of it like a scan system for peripheral vision and then a zoom-in system. So we went about doing that, and how and why we're in a position to be able to do that is based on a few different revolutions

that are happening in the space sector. It really is a burgeoning time right now that we've got to leverage. The first piece of it most people have heard of, which is to do with rocket costs coming down. Rocket costs have come down about fourfold over the time period that we've operated the company. Primarily this is SpaceX reusable rockets bringing the launch cost down, and it's been a tremendous service to the space sector, obviously enabling more. But a lesser-known aspect of what's really undergoing the space revolution is the capability performance of satellites. So we've taken satellites that are about the size of this room, literally a double-decker bus, and shrunk them in size, basically looking at the miniaturization of electronics and therefore enabling spacecraft to be much smaller for the same capability or equivalent capability. The upshot of both these, of course, are synergistic: cheaper launch and increased capabilities. But the thousand-times increase in capability per kilogram you put on the fairing of the rocket is, I think, the dominant factor that's changing the game in space. And the upshot of both of these things is all nothing to do with space. This has nothing to do with rockets or satellites or space of any kind. It's actually to do with data and it's all to do with the Earth. What we have effectively got is about 10 times or more Earth imaging data than we had just a few years ago, and we've got more than 10 times more bits flowing around the Earth for telecommunication satellites, primarily through Starlink. So basically all of it is about data. Space and data are married. This is just to give an illustration of the size of our spacecraft compared with Landsat. They're very small. Landsat is, of course, an incredible mission that has done the legwork in helping us to be aware of the basic changes that have been going on on the planet for the last 40 years now. And we really try to build off that legacy in just adding greater spatial resolution and greater temporal resolution. But just to be clear, these missions are still necessary because we calibrate our satellite data to these satellites. Actually more to Sentinel-2, but also MODIS and Landsat. And so this is our complete fleet today.

I'll spend a few moments on each satellite type. So the Dove satellites, we have about 150 of these in orbit today. It's the largest Earth imaging constellation. Each one has 3-meter resolution and has these eight spectral bands. They more or less mirror the Landsat bands and the Sentinel-2 bands, which are more or less the same in the optical and near-infrared spectrum. You can see the spectra here. And we image 200 million square kilometers a day. So the Earth's land mass is about 150 million square kilometers. We do all the land and we do some various parts of the ocean like the Mediterranean, the Caribbean, the South China Sea, anywhere there's hot spots basically. And just to give you a sense of the scale, we take 4 million 47-megapixel images per day. So it's a gargantuan amount of data. It adds up to about 45 terabytes in its most compressed form, although it's a fair bit bigger than that once we decompress it and do the analysis on it. And so we are generating gobs of data about the planet. And you know, we'll talk about AI coming up and the utility to extract out information from it. So we have all these different layers for every—so we have now on average on the Earth 3,000 images for each point of the Earth's landmass, and each has these eight spectral bands, useful for various

different functions. Then there's the Pelican spacecraft. This is our next-generation high-resolution satellite. We just launched two more last week on a SpaceX rocket. These ones, we call them 30 by 30 by 30 by 30. So they are 30-centimeter resolution, 30 revisits a day, and 30 minutes from task request to get the image back. So, super important for disaster response, various security applications. So we have now just got up to three of these birds in orbit and we just got our first light off these the other day. So, it's really cool. And then Tanager is our first hyperspectral instrument in orbit. And we're really excited by this one because, as a lot of people know, if you really want to understand things, taking spectra is one of the best ways of doing so. This is a 400 spectral band instrument. It was developed by NASA JPL and I think the first one cost about $50 million for the instrument. And now we have taken that instrument and figured out how to produce them more efficiently and build the spacecraft such that the entire cost of the spacecraft, including the instrument, the spacecraft, and launch, is about $6 or 7 million. So massive reduction. And the great thing then is that we can launch these at larger scale for important missions around the

planet. And this one, its first mission is—oh, I want to spend a second on this collaboration because I think it's really cool. It's a public-private partnership where NASA JPL, as I said, provided the instrument technology and we provided the spacecraft bus. Carbon Mapper is a nonprofit organization that developed this portal for looking at methane data, but they also provided the funding for this philanthropically funded mission. And it's a fun way in which public-private partnerships can happen that can enable us to speed up getting exquisite instruments in space. Those folks at JPL really wanted to fly a bunch of these missions. There wasn't the budget for that $300 million or $500 million spacecraft cost. Here we came along. We can do it a little bit cheaper, or about 50 times lower cost. And Carbon Mapper provided philanthropic funding because they recognized the need of getting these up there to do this carbon emission monitoring. It was a really cool collaboration. I was a little bit worried about impedance mismatch between us and JPL because we tend to operate very speedily, and I worked at JPL earlier in my career and it's less speedily. But it worked really well and we got these instruments in space. And its first mission is methane detection. So, and that only really uses a couple of the 400 spectral bands. There's lots of exciting other things that it can be used for. And here's the first light from last year, and this is just showing a few of the spectral bands.

So this was its mission, which is to look at methane gas emissions and to find them. So methane gas is one of the biggest climate forcing functions for global warming. And here's an example where we have started to release locations around the world where we see gas emissions. We've released about three or so thousand locations. This is an example where as soon as we put the location online, someone went and realized that they had a big gas leak. This is the first one that happened like this. This gas leak is about 7,000 kilograms an hour of methane, which is roughly about the same as, I think, 10,000 cars driving at full speed on the motorway, or about 100,000 or 200,000 cars if you take into account the duty cycle of car use. So we estimate that if you could do this about 100 times per month, you could take out about the equivalent of the US forcing function of all the cars in the United States. And so it's really exciting, the potential. And it's been even more sensitive than we thought. We got down to 66 kilograms of methane per hour. That's roughly the same as having four houses turn on all their gas ovens and letting that go out. So we can detect small amounts of methane by global standards. And the primary sources of these are gas facilities like gas pipelines and oil and gas infrastructure, but also all the cows and then the waste facilities. So we're looking at all of those and trying to help people stop that methane. It's one of the lowest-hanging fruits for stopping global warming. We also did a partnership with California to monitor these on a systematic basis over California, and they're paying for this data all over California. So the general aim of all of these satellites, one is sustainability. Obviously the sustainability applications per that methane example are kind of clear, but I could talk about a couple. And then digital transformation is industries around the world trying to become more efficient and effective. And then peace and security is our other big one, and we spend a lot of time, basically doing a global transparency maneuver which is having big implications for understanding events relevant to security. And I'll go into a couple of examples.

Can you play the video on this one? This was sticky—if you can play the video. I'm going to just show a couple of use cases. Here, if you press that button there—let me see if I can—Oh, did I do it? There we go. So in agriculture, it's not only that we can see every farmer's field every day and track, but one of our spectral bands, the near-infrared, can be ratioed with the others to enable us to detect the type of crop and the crop yield on every 3-by-3-meter pixel across the entire Earth's landmass every day. About 30% of which is agriculture. And plot agricultural yield over time and then help farmers to improve crop yield, and that's both good for agriculture, improving crop yield efficiency by about 20 to 30%. On the other hand, that also stops pressure on deforestation which is primarily driven by agricultural expansion. We work with utility companies like PG&E in California to try to prevent fires. So in this case, monitoring hundreds of thousands of kilometers of just their power lines every day and telling them where the brush is getting too close to the power line to do preventative work. About half the fires are caused by this kind of problem. We do some fun things like stopping marijuana. Actually, in a lot of countries—we just had another case in Panama and the Darien Gap that is leveraging our data to stop it. In this case it's legal in Hanover County but you still need a license, and it seems like a lot of them don't have a license. But it's actually an interesting way of doing, if you like, automated enforcement of permits across states. So various states use our data for that kind of purpose. This was another interesting use case in insurance, where instead of having to send people out to check on claims, they just do automatic checks on this. Swiss Re and AXA are the two big insurance companies, and they look at our soil moisture data. We have the amount of soil moisture in the top five or 10 centimeters of the soil. And this helps them—and basically if it's less than x amount of water for more than y days, they send out the adjuster instead of having to send out a person. And then of course my favorite use case of our data, which is on deforestation and trying to prevent it, of course. And this is just time-lapse of some of our data looking at deforestation, in this case in Brazil.

But this is a more hopeful story. This is a case where we automatically use an AI system to look across the whole of the Brazilian Amazon for new roads, because new roads are normally the sign of illegal deforestation. So an AI system looks for all new road starts across the entire Brazilian Amazon once every week, and it provides then alerts to the Brazilian federal police who then intervene in that situation. One of the best use cases I've ever heard. And last year alone they did 3,000 interventions, so about 10 per day, seized $2 billion worth of assets, and most importantly reduced their deforestation rate by 55%. So if you like, real-time information with AI analysis together with cooperative governments can go a long way really fast with this kind of tech. And the federal police, I'll just give you a sense, have 120,000 people looking at our data every day. So it's quite an extensive system down there. But that's not where we're stopping. On forests, we've now launched this thing called the Forest Carbon Planetary Variable. So this is our attempt to estimate not just whether the tree is there or not but the amount of carbon in that tree. So we've combined LiDAR data in local areas where we have it together with ground data of people using tape measures to measure tree trunks and species types to then calibrate an AI model that can estimate the amount of carbon in a tree. And it can do so pretty accurately. And so now we have this that you can basically pick your area of the Earth and it will tell you how much carbon is in all the trees in that area. And of course this is critical if you want to do carbon markets. If we want to integrate carbon or climate in general into our economic system, integrate that externality in. Right now we're not doing that a lot to do with the fact that there's not a consistent measurement system to underpin that. So we're taking away that excuse, or that's the idea.

Let me move on to security. We do a lot of work in security as well, and I think it's a really important mission of Planet that we're bringing this transparency to global events and helping countries to understand what's going on and bring accountability to those situations. So here's Ukraine, and we image all of Ukraine every day as we image everywhere else. And I'll share a couple of examples of how we were helping Ukraine from day one. And in fact, this is day minus one, when the Russian forces erected these bridges across from Belarus into Ukraine, and just the day that President Putin said that they were not invading. And from day minus one we were calling their bluff on everything that happened. This is them attacking power facilities. We work with Microsoft and the UN to estimate building-by-building the damage across all of Ukraine using automated damage assessment tools, which, by the way, subsequently we now use on a regular basis. In this case, in Lahaina, to do building damage assessment. Now we've got it down—the first time we did it in Ukraine it took two months. This now took two hours, such that quickly after a disaster, in this case the fire in Lahaina, we can get them data so that the medical and other emergency services can get in there really fast and determine which buildings to go into. But back to Ukraine, we also did a field-by-field assessment with our friends at NASA Harvest of the crop yields, the crop types in every field, what fraction was under Russian control, where it was being taken. Some journalists used this—investigative journalists used this to figure out that the Russians were taking grain and taking it illegally to Syria and other countries against the embargoes. So that then led to further escalation of economic sanctions and so on. The important thing here was Ukraine is supplying over half the world's grain to the poorest countries around the world. The bread basket—the World Health Organization, like Lebanon, 80% of their grain for feeding that poor country came from Ukraine. And so a lot of that was being stopped, and then our data was trying to help get that grain out, both for helping the Ukrainian people and others. War crimes analysis is one of the other use cases. So we basically have a digital forensic machine. So when, in this case, a mass grave was found, the question was what led up to that and can that be evidence of war crimes. And then letting everyone else know when things are happening, because we have various media outlets like the Financial Times and the New York Times. And when Putin said we don't hit civilian targets, we published the next day on the New York Times: here's the five schools and hospitals that were bombed in the last week, or what have you. So bringing accountability to warfare, I think, is a whole new thing. Now, it obviously didn't deter Mr. Putin, but my hope is that actually bringing this transparency will deter this kind of action in the future because everyone will get to see it.

On a slightly more fun topic, I want to talk about what we call digital public good projects. This is where—we're a private company, well not public, but it's a commercial company. But we have an important mission; it's a public benefit corporation. So we really care about the impact of our data around the world. Obviously you heard about our NASA heritage, that's our sort of ethos. And this is the first digital public good we did, in collaboration with the Allen Institute, which is to map all the coral reef systems around the world and warn of ocean bleaching events or fishing that is against the marine protected area, and so on. And actually able to identify the different types of corals under the water. Most corals are in very shallow water. I think 90% are under 15 meters, and at that sort of depth we can still tell the coral type. And then we revisit that on a regular basis, and you can just go to AllenCoralAtlas.org and look at your favorite coral reef systems and see what you can do to protect them. There's a dive-in to my favorite coral reef system which is in Raja Ampat, and there it overlays the marine protected areas where there's bleaching events for regular intervention. And the second one I want to tell you about is one on tracking deforestation. So in partnership with Norway for several years, we provided data to help track deforestation across 64 countries, with their forestry ministries getting our daily alerts of where deforestation is, a bit like I was talking about in Brazil. This is a slightly newer one. We launched it two years ago. This one is mapping all the global renewable facilities using our data. Again, this is AI on top of our data finding, in this case, wind energy facilities and solar energy facilities around the world to be able to map them. And it turns out several of the countries had completely different estimates of how much solar and wind energy that they had. So it's good to calibrate. And then this is the Carbon Mapper one that I mentioned earlier. These are all the methane leaks that it's been detecting around the planet from various facilities. How big they are. You can zoom into the exact building or facility that is producing these methane gas emissions. And that's all done as a digital public good for the world.

Also, I just wanted to mention since it's a scientific audience that we provide our data in collaboration with NASA to thousands of researchers across the United States and others around the world who do their own research on top of our data in these different areas. Now producing, I think, three peer-reviewed papers per day, and a total of over 3,000 now. You can see the steep rise in that. Our hope is that this can underpin not just science but also applications that help us to take care of the planet. So now I want to switch gears a little bit and talk about how AI is related to space. So a lot of people, obviously, pretty much every sector is being intersected by AI, and space is no different. But I think space is uniquely positioned to leverage the benefits of AI to do some really cool things. We started with CNNs, convolutional neural networks, but we're really moving into large language models or visual language models and foundation models that can lead to what I call Queryable Earth. But obviously this is quite the revolution. But AI is really about three things: it's about the compute, the algorithms, and the data. And you know, a lot of the big foundation model development by OpenAI and Anthropic and Google and others, they're really focused now—if you read their recent talks—on real-world challenges. They want to build these real-world models to tackle real-world challenges. And of course, if you want to tackle real-world challenges, you want real-world data. So we feel at the center of this revolution that's going on. We have this powerful dataset that can potentially help solve real-world challenges. Our data is hard to play with because we've got, you know, 45 terabytes a day. So most people are like, what the heck to do with that? But AI can make it much more accessible, right? And just like ChatGPT helps you access all of the information of the text of the internet primarily, the question is: can you imagine what could be done if you could have a queryable interface, a semantic interface to the Earth? And so that's sort of where we're heading

next. And the basis of that is the fact that we have these 3,000 images for every point on the Earth's landmass. So we can train algorithms on what's normal, what's not normal, and to detect changes. In many ways, what AI is doing today is—if you're typing in a sentence, it will guess what the next word in the sentence is, it's guessing the next token. And a sort of equivalent statement in our data stack is guessing what the next image is going to be, and therefore where's the next refugee challenge going to be, where's the next deforestation event. So predictive analytics, I think, is going to take off tremendously over the next coming years because of this data archive. And the other reason we feel in the center of this revolution is because we're in the center of this revolution physically and geographically. A little area of San Francisco seems to be where it's all taking off. I know everyone thinks that Google is in the South Bay, which it is, but their Gemini team is all up in San Francisco. Everyone's knocking San Francisco, but this seems to be an interesting development. So I'm going to show a few examples of AI using our data. Can you see the ships? There they are. And this detected a plane as well. So this was using CNN, convolutional neural networks. And we'd shown it thousands of examples of planes or ships in order to get this working. And that's certainly been helpful. And a lot of people use that, and we did that with planes, ships, roads, buildings, various things. And we can search the world for things like that at large scale now. But that's a particular thing that's taken a particular training model. Now the question is: can we go to more generic

Is just one more example where we did roads and buildings for the whole Earth, which is cool. And there's a lot of roads that are there that aren't on Google Maps, it turns out. But the question that I posed to our team about seven or eight years ago was, what if we could look at every single object, detect every single object on the Earth every single day, and turn that into a database that you could search for all objects over time? Then you'd have some sort of version of queryable Earth. So I gave a TED talk about that, and you can imagine asking semantic questions like these, both of the current Earth or predictive even.

And so that's what we started building, and this is a collaboration we did with a partner where it was a first use of large language models on our data, which was really cool. If people remember, there was a balloon that was flying over the United States. And some people at NORAD called us up and said, 'Hey, can you find that balloon?' And we're like —

Yeah. I think they knew where it was, but they were also interested in knowing where we thought it was. But anyway, so we not only found that balloon but managed to trace it back through time because again, we have the layers going back. We never delete a layer. And we traced it all the way back to Hainan Province where it was launched and found it within an hour and a half of its launch.

So digital forensics of the whole Earth start to become possible because this thing, by the way, had never been trained on balloons. That's the distinction of this new development. This thing in fact was sketched a balloon on a piece of paper and shown that, and then it just found it across our imagery. So things are changing to this single-shot or no-shot learning. And I want to just show this very quick example I did literally the other day. If you show images, just satellite images, to these large language models like Claude or OpenAI's ChatGPT or Google's Gemini, it can actually just start doing analysis out of the gate.

So you can go zero to some sort of analysis of satellite imagery in literally seconds and at scale. So I think the three things this is doing is enabling the speeding up of analysis of satellite data, the scaling up of it — so suddenly you are able to ask questions like that previous one was across a significant part of the world but certainly the US, Canada, and China. And the third thing is democratizing access because anyone can look at an image, put it into one of these LLMs and say, 'Tell me what's going on, give me a graph of land use over time,' or in this case estimate tree canopy cover over time. And it will have a go, and it might not be super accurate, but it's going to have a go, and it's a good way to get from zero to something.

And most organizations we work with have teams of people with PhDs in satellite imagery processing, but most organizations don't have those skills. And so we think we can open this up decidedly. So my prediction is that we're going to move towards an ability to search the Earth just like you can search the internet now with ChatGPT. And imagine the power of being able to do that. And so I wanted to close out with just a couple of slightly more philosophical things because — well, this is a philosophy and science club, so here we are.

You know, I think we're on the verge of some really important Copernican moments as a species. The first is racing towards us somewhat involuntarily, it feels — AGI. We're giving birth to superintelligence right here on this planet in the next 10 years. The other couple though perhaps are less known that I think it's really likely that will happen. The first is that we're going to have first contact with animals. And what I mean by that is deciphering animal language with AI and being able to understand what they're saying to each other. And just imagine — whales have been speaking to each other for 70 million years. They might have some interesting stories. And it might expand our empathy circles to care about animals a bit more if we understand their social patterns.

And then there's the first detection of life off Earth, which I also predict will happen within 10 years. A lot of people can see that coming because of detection of interesting atmospheric spectra that we're now being able to do of planets all around our galaxy. The nearby ones — we've got thousands of planets to look at. We've now got telescopes coming online that can do spectra of these, and one of them is going to find an interesting spectra that we're going to go, the most likely explanation is probably life. And we had an earlier version this year where we thought maybe there was a case — it doesn't look like it probably is, but I think that will happen with more definitive nature.

And to think that we're on the verge of these three moments, all three decentering humans from our anthropocentric ways, I think is really compelling and exciting. And the way I think of Planet fitting into that context is holding up a mirror to help us take care of the planet and by that means helping life understand itself. Or even, you know, I often think that humans are definitely conscious at an individual level, but if you think about how we're taking care of the planet, not very conscious at a collective level.

Collective consciousness is something that I think we need both great intelligence for, but you also need great sensor data. So that combination — you know, babies wake up and become conscious in part by having sensors and actuators and interacting with the real world as well as the brain material. So now we have AI coming along together with this massive amount of data about the Earth, and perhaps this can help humanity to become a little bit more collectively conscious or at least a bit smarter. So with that speculation I'll leave it, and thanks very much for everyone's time.

Well, actually this is changing very fast. So one of the most exciting developments I didn't touch on was the work of embeddings. So there's these incredible developments in embeddings which basically project the images into these vector spaces but somehow enable extraordinary efficient searches. And that is the basis of that balloon example.

Google DeepMind just came out with a paper on this which is really exciting, and Meta did too. And basically that is going to enable the compression maybe 100 to 1 or something of these data sets to make them basically queryable. The other day our teams did an incredible job looking at an entire basemap of the whole Earth and searching it in under 10 seconds. So you can give it either a word input or an image input, and it will search the entire Earth with it in under 10 seconds. Just imagine the power of that, right? It's really cool.

Great. Even better.

That's a great question. I can't wait to try. I think there's a question of the radiation going through the Van Allen belts and how well the electronics will survive. But I mean, traditionally our approach to that has just been redundancy. So throw up a few more and probably most of them survive. It turns out, you know, a lot of these electronics survives better in these radiation environments than you think. So my gut would be not very much. Obviously you need different propulsion systems depending on your maneuvers and stuff. But I'd be willing to put a wager that if you put 10 Doves or Pelicans around Mars or something, they probably just work.

Yeah, so most of our satellites fly about 400 km. That's not that low. It's low, but you can fly lower. I mean, we're looking to operate our Pelicans — we name them Pelicans because they're low flying and pelicans get lift off the water when they fly really low. And so we were sort of thinking maybe we should get lift off the atmosphere. Anyway, that's a loose analogy. But it only really becomes a problem when you get down to sort of 300 km. That really becomes hard. But we have ion engines that are keeping them at that altitude so that we can keep operating them.

But also, we have a design life that's much shorter. You know, one of the things my colleague was always pointing out also with the phone is that just like you don't want a three-year-old or five-year-old or a 10-year-old certainly phone in your pocket, you don't want a 10-year-old satellite in space. Because the electronics, the sensors, everything gets obsolete. We call it strapping space to Moore's law. Every launch we're putting up the next generation sensor, the next generation processor. So in those Pelicans that were launched last week, we have Nvidia Jetson Orin processors — it's the fastest GPU in space. So that we have more processor in space. We don't even know what we're going to use that for exactly yet, but we're capabilities-driven. And I'm sure there's going to be algorithms that we want to put at the edge to do onboard compute to get certain data and answers back faster.

So by doing that we produce far more data every time than trying to keep the satellites in orbit for longer. And they just naturally decay and hit the top of the atmosphere and burn up at the end of life.

It's a great question. Not a huge amount yet, and a lot of that's because we haven't heard any complaints from the astronomers about our data. Basically, our satellites are very small and so they don't tend to cause the same challenges that the Starlink satellites have been causing for astronomers. Starlink is about 10 meters long. Ours are about this big. So much smaller, but more mighty.

Yes, in important ways they are. So our data is being used not so much in the kind of way you were suggesting, but more in domain awareness. So battle damage assessment, figuring out where people are doing what. And the most important thing about it is that we are ensuring that multiple actors can access that data as well.

Yes, I mean, well certainly news media, NGOs — so in Ukraine there are many NGOs that are using our data, and the New York Times and the Financial Times and the BBC use our data. You would have seen it all over the press in the case of the recent strikes in Iran or wherever, bringing accountability to what's going on. And so we think that's an important principle because we think accountability drives towards more peace. Historically warfare has been typically driven when countries make mistakes, and that's normally when they don't know about things. Classically, the Cuban Missile Crisis was when the US didn't know the Russians put missiles in Cuba and then discovered them, or vice versa. When the US put missiles in Turkey, that's when things almost got hot. When everyone knows what each other are doing, that tends to deescalate things. So we believe in that theory of change for security, and we lean into it.

Of course. Yeah, I mean, so we work — the basic answer is a lot. We work a lot with NASA, ESA, and our friends there, integrating their data sets. As I mentioned, our Dove satellites and all of our satellites are calibrated to Landsat, MODIS, Sentinel data sets. That's how we calibrate the spectral bands. And then there's most importantly interoperability in the sense that thousands and thousands of researchers, probably hundreds of thousands, are using both data sets and then looking at interoperability, using both data sets all the time in practical applications and science. So we'll continue to do that. We'll continue to work with NASA, see if we can do missions with them, for them. And we'll continue to try to calibrate our satellites so that they are synergistic.

Right, and I hope that we will do that in the future. In general, we're moving more — less user-driven, more AI-driven. So I mentioned how we have this peripheral vision with the Doves, we then have AI that's enabling us to figure out change, and then we have a high-resolution system that can zoom in. It's very much how the eye works. You have a sort of medium resolution in most of your field of view, a high resolution in the center — as predators, we have those little high-resolution piece — but we have a very good visual cortex and that makes it look like all of this is high-res. And then we're very good at change detection, and if something changes, we take a high-resolution snapshot. And in that way it looks like we've got a high-resolution image all the time in our field of view. And we're trying to do that basically for the Earth in combination.

I think there is also user-specified things that you could get into as well. I think that's interesting.

I get it. I think it's a good idea. We don't do that at the present, but I think as I said it's more algorithmically driven. But I think we could do.

Yeah, is that a SAR mission or is that an optical mission? Is it synthetic aperture radar? So that's quite a different modality. That is — then obviously we're relying on reflected light, that has to transmit its own beam and then get the receive beam in order to make measurements. I think those are very complementary assets, right? You want both optical and SAR often to determine features. Optical doesn't work during clouds or at night. SAR, on the other hand, tends to be much more expensive because of the energetics and you have to have a lot more power on board. The duty cycles are less and so on. So they're complementary in the capabilities that they offer.

Well, it depends on the resolution of the imagery you put in and also how good the algorithms are. Right now our 3-meter scan doesn't pick up drones very much because drones tend to be smaller than that. The 50-centimeter data can pick them up in some cases. The AI systems are quite good. If you can visually see it, probably the AI can see it as accurately, if not more, because sometimes they're using the spectral bands that you can't see and other things to figure out features. But it really just depends on the resolution. We've certainly seen a bunch of drones, but generally that's not what we find all the time.

Well, our mission is a little bit different from OCO. The hyperspectral mission Tanager can help with some of that aspect, but it hasn't got the wide swath that OCO has. I really hope we continue to have that. Once again, this is complementary data sets. You want the scan and the calibration that OCO is doing and you want the zoom in that the Tanager does that enables you to pinpoint a source of carbon or methane emissions. So you want both ideally. So ideally we would continue both.