Boz To The Future Podcast #23 - The Future According to James Cameron

Yeah. You know, and it's spatial data, spatial data over time, right? I can't help but imagine that there's an answer there. It might be in light field. It might be in plenoptic, but there's some kind of answer there. We got to put this in the podcast.

Well, I think people know me as a filmmaker and some people know me as an explorer and documentarian of mostly ocean subjects. A couple of other things. We've done some documentaries on a few other things. Probably don't know I'm a farmer.

Well, it's all organic farming and it's investigative farming, looking at new agronomy around organic systems and organic crop rotations. I just believe the future of food is going to be a big deal and it might be one of the things that takes us out if we don't really plan ahead. I'm worried about peak phosphorus. Using organic rotations, you keep the nutrients in the soil, you replenish the soil. You don't have to bring in synthetic fertilizers or stuff that's got very long supply chains and very high carbon supply chains.

Exactly. Which is hugely costly and it's running into some limitations. Alfalfa is a very interesting plant because it has a very deep root system. Most plants, wheat, corn, whatever, they extract their nutrients from the top six or eight inches of the soil, so you have to constantly replenish. Alfalfa is going down 40 feet. It's pulling up what I call paleo nutrients from thousands and thousands of years ago, so it's not going to deplete the soil quickly. Then you can use it in other things, put it into green compost and grow vegetables with it.

I just always loved 3D movies. I thought it was right at the point where HD was starting to show the promise of replacing films. This would have been in 99, 2000. Film cameras have limited 3D. They wind up these really massive camera systems because the film plane can only get so close together. You've got wheels, feed to take up, magazines are big, it all scales up. Whereas a digital camera ultimately all you need is a lens and a sensor. I intuitively saw that. They still were fairly big. In 99, I went to Sony and said, I want to build a 3D system. I need you to make the camera smaller. It could be longer, it could be taller, but it needs to be narrower so I can get the two lenses side by side. I thought side by side was the answer. But if we came in to try to do a close-up on you or on me, we'd have to be able to shrink that interaxial distance. The sensors can't physically overlap, and the lenses can't go through each other. In CG they can, but not in the real world. Eventually I realized that to do proper cinematic 3D, you needed a beam splitter so that you could actually get those lenses to overlap spatially. I went down the rabbit hole of stereoscopic photography. It was right at that jump to digital and that's what drove it. All of a sudden it was a wide open field. With digital cameras, you could do things you could never do with film cameras. I had made a 3D film for Universal called T23D. That was a fixed base kind of thing, purpose shot just for that attraction. The cameras were the size of a refrigerator. I thought we can do better than that. What I wanted to do was go out and dive Titanic again because I'd made the movie a few years earlier. I wanted to take 3D cameras out there and shoot an underwater expedition. It's great to have a deadline and a purpose, not just sort of make stuff up. We wound up cracking the code on how to not only create a digital 3D camera, but how to put one in a housing and take it down 12,000 feet.

Now you've got to have a pressure sensitive vessel that's able to deal with it, that's not going to implode and take you out. Then you need a pan tilt so you can operate it. I didn't want to just poke it through the viewport. I wanted it out there in the water column, able to look straight down, look up that hole, look up six stories up the cliff of the Titanic side. Thinking cinematically. The motif is getting the technology prepared in order to do an artistic thing. That was true with CG, that was true with the 3D. People just accept that digital cinema is the thing. There was a point in time in 2003, 2004, where digital cinema was not obvious. The exhibitors didn't want to spend the money on it. They already had projectors. 35 millimeter was really good. The value proposition to the audience was exactly what you got before, except it's digital. That's not enough to justify the capital expenditure to replace all the cinema. There was nothing to sell. But I saw an opportunity. I got in early with the Texas Instruments guys. There was a guy named Doug Darrow who had developed this amazing mirror chip, which is the heart of all digital cinema to this day. I said, Doug, make sure it's 3D compliant. Make sure your entire chain, your driver, your server, everything is 3D compliant. The thing would switch at a really high rate, 120 hertz or something. I said we could do multi-flash at that rate and decode using a multi-flash system. He said okay, I'll build it in. That was the catalyst that got digital cinemas out there was 3D, because they could charge more for the ticket. Theaters could say, here's a new value prop for you the customer, it's 3D and you just got to pay a couple extra bucks. The studios were happy because they could ship a digital cinema print the day before the film opened as opposed to three weeks of living hell where the labs were working around the clock. Most people don't even remember that world, the photochemistry world. Now it's just kind of instantaneous delivery.

It was a little bit of a chicken and egg proposition. When I saw what was possible on The Abyss with the single scene that we did in CG, which was the water tentacle, then I did Terminator 2 and I based the entire story on this adversary who was a liquid metal guy who could shape shift. Still one of the all-time greats. Stan Winston, who I worked with, did the physical part of it. He made the makeups and the gags and all that sort of thing that I could photograph, but I still had 40 shots. That was a lot of time. Stan and I came out of that saying, we need to do this digitally. So we founded a company called Digital Domain, which was the world's first all digital VFX company. Our third founding partner said, 'And we'll put the optical department over here.' He used to run ILM. I said, 'Dude, there's no optical department.' He said, 'Well, you got to have an optical department.' I said, 'No, that's the whole point. We're going to leapfrog over that.' When we cracked the code on digital comp, we were going to be ahead of everybody else, including ILM, which was true for a while. Of course, everybody caught up very quickly. You don't maintain these market advantages very long, but maybe a year. We cracked the code on how to do digital image creation, composite animation, all that sort of thing and do it all under one roof. That was Digital Domain. Then I thought, how do I kick these guys in the butt to go to the next level? Creature, character, human-driven performance. So I went off and wrote this little thing called Avatar. I brought it in and showed it to everybody in 95. They went, 'We can't do this.' Every single shot, every plant, every blade of grass, every insect, every character, close-ups of humanoid characters. They said, 'We can't do this.' I said, 'If we start now and do our procedural muscle rigs and our image based facial capture and all that, when can we do it?' They said, 'Maybe long after we're bankrupt.' So I shelved it for 10 years. Then I saw the second Lord of the Rings film and how far they'd gotten with CG performance-based stuff with the Gollum character. I said, I think we could do this now. I was long out of Digital Domain at that time. We founded a little group together to explore how to do it. We went to ILM and Weta and worked with them to figure out how to do it. The technology was driven by the desire to tell a story, to see a thing, put on a show. We had to create a lot of new technologies for that, including the image-based facial performance head rig system, which is pretty ubiquitous now.

Exactly. Because you can capture an actor but the whole point of doing capture is to apply that to a character that is not the actor. Otherwise just film the actor. There's a transform there and then you have to have an algorithm or a rig for that particular character. Once you get deep into that you learn how to simulate the surface that would have been deformed by all of those muscle systems. You don't actually create the muscle system, you create the effect on the surface of the muscle system.

That's right. This is the crazy thing about where we are right now in the world. We've got 38 years since The Abyss. The Abyss was one of the first cinematic uses of soft surface CG. After almost four decades, it's a very mature art form and it's very controllable and iterative. Then you've got these gen models or even just kind of machine learning models that don't do it that way at all. They don't do it that way at all. It's not that it's history, it's just that's not the method. You can go straight to a result that's a condensation of everything it's observed up to that point. Meaning it's probably actually trained on a whole bunch of thousands and thousands of faces and then it just modifies it based on your face. Using the common generality that we kind of expect because we're all human.

Documentaries would always have lots of lens flares because a guy's running around with a 16 mm Bolex backstage at a Rolling Stones concert or whatever. Can't control the lighting. Hollywood film making was perfect. Never had a lens flare unless you were doing some sunset shot and you were being arty. Then filmmakers wanted to have a sense of verite in their narrative films, so they started introducing lens flares. Then we come along and we've got these pristine CG shots that can be perfect. It's like no, put a lens flare on that. So it feels like it's real.

But that's being an artist. Computationally, whether it's CG animation or a gen model, it can be perfect, but whose version of perfect? You're the artist. You're going to make your version of what you want it to be stylistically, which is why there always has to be an artist in the loop. There's no arbitrary bureau of standards version of a story. It's always interpretive.

The physics, very unyielding the physics, I would say.

I've been thinking a lot lately about the inception of any project that I ultimately commit to, whether it's an engineering project or an expedition which pulls along engineering projects with it in order to accomplish those goals, or a movie. It's when I find myself thinking about something that I can't solve for too long. Then the solution occurs to me, then I have to do it. Because I need to know if I was right. I can't just imagine stuff and assume it would have worked if I had actually done it. I'm hooked. I have to go do it. You have to infect other people with the same curiosity and problem. You have to pose the question to them and let it burrow into their brains for a bit. Then they're going to come back to you. You don't do anything by yourself. Complex engineering problems like building a vehicle, it's layer upon layer upon layer of electronics and control systems and electromechanical systems, thrusters, pressure boundaries, optics and cameras. It's like saying, I'm going to build a jet fighter. You're not going to do it yourself. You need a team. I find that I somehow managed to pre-select, I guess, with my pitch, people who like hard problems. It's a little bit like when Shackleton was populating his expedition team to circumnavigate Antarctica, which failed miserably, but it was actually kind of quite heroic the way he kept everybody alive. His newspaper ad said something like, 'Antarctic expedition, some danger, survival not guaranteed.' That's how he recruited the guys he wanted. Selection bias. Anybody that was cool with that.

The first time I put one on and I saw the quality of the image and the brightness and the dynamic range, of course I knew all these MR devices have one thing in common, which is that they're creating a discrete left eye, right eye image. A lot of the problems associated with stereo display in theaters automatically go away. They're just solved. You're not giving one image to everybody and having it decode locally at the face with some form of glasses, polarized or shutter or whatever. You're getting full brightness to each eye. You're not knocking out half of the photons with some kind of filtering device. Brightness is a big deal in theaters. The heartbreaking thing for me is, we have a perfectly calibrated theater that we work in for our final authoring steps of grading and brightness, color, dynamic range, all that sort of thing. It looked great. We're just so self-congratulatory. Then we go out into the real world and see how 99% of people are seeing it, or 99% of theaters, I should say. Less than a quarter of the light. The Society of Motion Picture and Television Engineers has a standard for screen brightness, it's 16 foot-lamberts minimum. 3D hits a quarter of that or less, 3.5 foot-lamberts is pretty standard out there. When you lose that amount of brightness, you lose a lot of the perception of dynamic range. It doesn't matter how good your blacks are if your highlights are only 50 nits above that. It just looks dim and dull. We're giving you spatial cinema, but we're taking it away with the other hand. Now when I see the HMDs, the new generation ones, including competitor, it's not really a competitor because you've smoked past them with a— Oh, you say it. I didn't say that. I'm saying it out loud. It's like, okay, a door opened. For me, it was like the heavens parted, light shone down, and there was an angel choir singing. This is how people can see the movie the way I created it to be seen. The scale of the display is subjective. If you fill your field of view within the headset and you're converging it out there on a distant plane, you feel like you're in a theater. I was just looking at some Avatar 3 material that we're going to share with some of the execs later this morning and thinking, 'Wow, this looks great.' This looks at least as good as a Dolby Vision Laser Cinema, which is the ne plus ultra in an actual group theater. To the podcast listeners, if you want to experience the films the way that they were mastered, Dolby Vision laser, look for that. That's your best chance if you're going out to the cinema. IMAX Digital 3D is pretty good, but not in all theaters. Dolby Vision has an absolute rigid standard that they hit. They're like you would have QC on every device that goes out. That's how they think. They're tremendous technologists, doing tremendous work both in audio historically and in the video space as well.