Extended: Ilya Sutskever

Well now we're going to fast forward to today as we think about inventing the future. This next panel we're going to look at revolutionary changes that are underway today in the world of computing. So every day machine learning and AI, as they were just mentioning, are center stage in the public spotlight. There are stories that are positive, there's stories about the perils, perhaps similar to other inflection points in computing history. We can see a lot of momentum and change, and the way forward may seem uncertain. A recently released report by the Stanford Center for Human-Centered AI showed the dramatic increase in significant machine learning systems over the past decade. Incredible, right? You can see that the locus of activity has shifted to industry. 32 more than 10 times significant machine learning systems have been deployed in industry compared to three in academia last year. This report also found that AI research publications are on the rise across the board. The number of AI publications on subjects such as pattern recognition, machine learning, and computer vision have more than doubled in just five years. At the same time, AI is more widely applied, concerns about the misuse are rapidly rising. There are calls for new kinds of controls, possibly policy and regulations. With all this in mind, we're excited to welcome for our next panel two computer scientists who are literally pioneering research labs to discuss their views on the momentous advancements that they're helping lead, where they see it going, and what might be on the horizon. So as is our tradition, here are our next speakers. Five numbers: first up, Ilya Sutskever. 2012 co-published AlexNet, three years on the Google Brain team, years at OpenAI, 20 years working on AI, and four GPTs. He's co-founder and chief scientist of OpenAI, which aims to build artificial general intelligence that benefits all of humanity. He leads research at OpenAI and is one of the architects behind the GPT models. Prior to OpenAI, he was co-inventor of AlexNet and sequence-to-sequence learning. Please welcome Ilya Sutskever.

Next, Eric Horvitz. 3,510 number of transistors in the first microprocessor. Programmed it was a 6502, by the way, in 1984. $150 was the stock price when he started at Microsoft. 1998 was the year he developed his first ML-powered email spam detector. 102 number of interns mentored at Microsoft so far. And 1,500 KHz was the radio frequency celebrated at birth. There's a story behind that. So he's Chief Scientific Officer at Microsoft, where his research centers on challenges with uses of AI, and he's a member of the President's Council of Advisers. We're so glad to have you here, Eric. Welcome.

And here to moderate our conversation is Diane Sane. She's a professor of computer science and adjunct professor of mathematics at TUS. She was appointed to the National Science Board in 2008 and served as its chair for four years until 2020. She currently serves on our own Board of Trustees here at CHM. Please welcome Diane.

So thank you, Ilya and Eric, for joining us for this conversation about research. We want to divide our conversation into two parts. We'd like to start by talking about connecting Alto's legacy to your current research that's on the edge. But to kick it off, I'd like to turn to Eric first and ask you: how did the Alto affect you personally?

The Alto actually had a deep effect on me. I walked into a computing room at Margaret Jacks Hall in the early 80s. I was just coming into Stanford from doing a biophysics undergraduate degree, focusing on neurobiology, and I was stunned at seeing something that I thought, as a mainframe programmer at the time, was delivered from the future. And that artifact led me to really pursue who is the team behind this, what was the research story, and that brought me to the names JCR Licklider, Vannevar Bush, Doug Engelbart. It really set me up and framed the research I came to do, which intentionally focused on Licklider's dream of pursuing human-computer symbiosis, as he called it.

That's great. So now for both of you, how are your organizations' current cultures and environments? How are they framed, and how does that maybe compare to PARC? I'll start with you, Ilya.

Thank you. So Eric, what would you say?

Most of my time at Microsoft has been spent at Microsoft Research. It's delightful to see Rick Rashid here tonight, who founded Microsoft Research. Two years ago, I moved into becoming Chief Scientific Officer after serving as director of Microsoft Research for a number of years, and before that, director of our lab on the main campus at Microsoft. In my current role as Chief Scientific Officer, I lead what's called the Applied Research Council, because over the years, while Microsoft Research was in many ways in 1991 when it was formed maybe the only deep research team at Microsoft, over the years we saw teams like HoloLens, our search teams, cloud research growing across campus. So now we have a nexus that we actually coordinate through something called the Applied Research Council, which is very interesting for me to co-chair. But Microsoft Research, Rick I think had it right when he put as the first mission part of the mission statement: advance the state of the art. There was no mention of Microsoft in that statement, and that was taken to heart in thinking out of the box. In fact, I don't know if Rick's here tonight, but he remembers in my first year at Microsoft Research, I just came up from Stanford and I was really excited about many things, including shipping stuff, changing the world to Microsoft's current applications and services. At my first review with Rick, he said to me, 'You know, it's all very exciting, Eric, what you're doing with all these product teams, and I'm sure they're very thankful, but are you sure you're spending your time in the right place?' The fact that the lead at the time, and I inherited this spirit when I worked with people when I came into a senior role, was to really think deeply and unleash creative powers. Along with that notion of thinking out of the box and not being strained by current products or current understandings, was this idea of openness. One of the deepest principles at Microsoft Research was there are no controls on publications or your interactions with researchers in the open world. This is reminiscent of hearing Alan Kay talk about the ARPA community, because we're all part of a very large community. Even at a company like Microsoft, we're connected deeply with other companies, with universities, and having that community be part of our thinking process was always very important. This idea of an open research lab in many ways. So I think the openness, the creativity, being pushed to think out of the box. One of the ways I would justify some of the things we were doing in recent years was very Alto-like: create something that's magical and way out there, it's way from the future, and you want to talk to Microsoft folks about it, bring it back, pull it from the frontier into the present, and then we'll work on the bridge.

Okay, so we're going to think way outside the box and think of where we sit today. What do each of you see as the most important or maybe the most audacious goals for computing now, and why?

Okay. So do you have a sense of, in the spirit of that goal, then how do you spur people to think broadly about what is serving humanity or what is possible?

Thank you. Let me resonate deeply with Ilya's comments just now. By the way, I'm sure we both remember a meeting maybe four or five years ago, an early Microsoft-OpenAI meeting in a conference room where at some point in time, Ilya got up to the whiteboard and he drew the plan for general artificial intelligence. And we just sat there with our mouths open, because I brought with me a set of serious Microsoft researchers who were like, 'Okay, can you draw that again? Can you show me what that piece means?' And Ilya, he's right here, very serious, he said, 'Here's what we're going to do.' Let me just mention, I mentioned earlier that I came to Stanford from undergraduate training in biophysics with a focus on neurobiology. I spent two years in darkened laboratories looking at oscilloscopes, what neurons were doing in small animals, trying to figure out how this relates to thinking, to human minds, to the foundations of intellect. Of course there's a big gap there, and I decided in heading to Stanford that the fastest way to get insights about what might be going on in our own minds was through computation. I made that call. So for me, the audacious goal in computing today is to understand deeply, for humanity to understand deeply, the foundations of mind, of human intellect. And many things will come of that, of course. Now I think that Ilya and I are very much aligned on that. Right now, let me just mention a couple of shorter-term goals briefly. I think this gets back to comments we heard earlier from Butler and Alan Kay about what's going on with the other thousand orders of magnitude. And let me just say that I would love for everybody in this room, if you haven't done so yet, to read through Vannevar Bush's vision 'As We May Think' and JCR Licklider's vision of human-machine symbiosis. We're nowhere near the vision that those ARPA funders were so excited about and why the lab, why PARC existed. I've made this part of my research agenda, a chunk of it over the years. But think about this: systems should be reasoning about how to complement people in a deep way. They should understand our thinking processes, understand the gaps in our reasoning, how to best complement us, whether it be in creativity or logical reasoning, or synthesis, abstraction. If you read the beautiful writings of Licklider, who was also a psychologist by the way, interested in computing, the very moment he saw his first interactive computer, which was the Whirlwind machine, he had this gigantic vision like these machines someday would be scientific apprentices. They would be doing the work to help human scientists understand and extend their thinking in ways that you can only do if you understand human thinking. So I see that as a near-term audacious goal, to really continue on and push on the vision that Licklider laid down. That's the vision behind the Alto, which is part of the way there and it was a big jump at the time. And then just to mention a third goal, I think Ilya and I would agree with this also: even today, we're seeing incredible opportunities to apply various forms of AI, not just deep learning, but of course the advances of deep learning are really impressive for the biosciences and healthcare in particular. I think we're going to see a huge transformation of medicine, and we're going to see an incredible acceleration in bioscience over the next decade.

That ties very well into the next question. I have a three-part question about the future. First, what do you hope will happen?

Okay.

You know, we're very much aligned. Look, it's clear to everybody in this room, especially given the recent developments, many thanks to the work that Ilya has been leading, that AI will be, in various forms but largely expanding out now from the deep learning research program and expanding beyond that, calling back to other aspects of AI as well, will be extremely disruptive to humanity. Now disruption is not a bad thing, but it raises our anxieties, it makes us apprehensive. And what I'm hoping for is that it all goes well. And we all know the touch points: the influence of AI on jobs and the economy, how these systems will affect our education and our sense of what we do in life, how we spend our time, our sense for the uniqueness of our identity as humanity when it comes to, for the first time in history, intellects that are kind of at the level in many ways that we had thought was a unique aspect of what it was to be human on Earth. And so I think with guidance, it will go well. And one comment I'll make is I've never seen, I don't think in history, we've had so much deliberation and reflection across sectors: the governments, private sector, civil society, thinking deeply and proactively to help and hopefully come together to assure that things will go pretty well.