S2, EP6 - Dr. Prith Banerjee - ANSYS CTO

Sure. So first of all, thank you very much for inviting me to this. So I started my career in academia. I got my PhD in electrical and computer engineering from the University of Illinois Urbana-Champaign and I started as a professor at Urbana. I spent the first dozen years going through the ranks, became a full professor, and I was the founding director of the computational science and engineering program at UIUC. Illinois is a big area. So my last two years at Illinois I was a founding director of computational science and engineering, which is the field of computing, high performance computing, using HPC to drive science and engineering, so computational physics, computational chemistry, computational electromagnetics, all of those. And as it turns out, 30 years later I have landed up at Ansys in this job, so that's sort of the connection. Then after Illinois I went to Northwestern, I was then at University of Illinois Chicago, so hardcore academic for about 20 plus years. After that I made a hard turn into the corporate world. I was head of HP Labs, and at HP Labs I was to lead a lot of work on high performance computing used to build... I was also CTO of ABB, a power transmission company based in France. About six and a half years ago I joined Ansys as a CTO, so this is my third CTO job. And what Ansys does is we are the leading modeling and simulation company in the world. We take the world around us which is governed by the laws of physics, and we take that physics which is explained as second order partial differential equations, and we solve those physics through finite element methods, finite volume methods, using things like Fluent which is our fluid code, and things like Mechanical which is our structural code, and things like HFSS which is electromagnetic code. And my role as CTO is to look at how does AI/ML improve simulation, HPC, how do you use HPC to accelerate simulation, what do you do with cloud, what do you do with platforms or digital engineering. So that is, I have the coolest job in the company, looking at the future of simulation.

So since you are asking a career question, I actually bypassed one part of my career. So I've actually had near three phases in my career. I was in academia for 20 years, but in between academia and the large corporate world I was in the startup world. I did two startups: one was Exelchip, one was B-chip, and these were companies started out of technologies from the university, one from Northwestern and one from University of Illinois. And I did those while in universities. You can actually go on sabbatical, so I left, I took leave from the university, did my first startup, came back to the university, the second startup, came back to the university. And literally the reason I went from the academic world to the corporate world is because in academia we do research problems where we are trying to really understand what are the absolute fundamentals of technology, and you work with graduate students. I have had in my 20 plus career in academia 37 PhD students, 40 plus master students, with whom I have published more than 350 technical papers in IT conferences and IEEE transactions and so on and so forth. So that's the world of academia where you are researching, you're discovering new things, and you're publishing that work in the latest journals and conferences. It's all about creating new knowledge and then transferring that knowledge that you have created, training undergraduate students, graduate students, and so on, which are the workforce for all of us in academia and in the corporate world to do. But what academia does not do is we don't build products. And literally, Neil, the reason I did the startups was I was frustrated that I was doing all this work, 350 papers, 10 plus patents, doing all kinds of stuff, but nobody cared, nobody gave a damn, because it was not showing up in any product. So when Exelchip was actually created, I ended a DARPA project called the MATCH compiler, and the DARPA PM said, 'Prith, this is really awesome, you should transfer it to...' The only way to really commercialize this software was if I were to do it myself with my graduate students. So that's kind of why I started the first company, Exelchip. So startups, what they do is they actually take a really new idea, something that the world has not seen before, and get laser focused on that idea, and they bring that new product to the market. And I did two of those startups myself, and then I came to the large corporate world of HP, ABB, and so on. But what I have found is the large companies, they don't have a single product. Like Ansys, we have 70 products in simulation. We have to decide what features should I have in the next release of Fluent, the next release of Mechanical, and so on. But the innovation that happens in the corporate world is more incremental. It is what I call Horizon 1. I have a product, so I used to work at HP, you make computers, so the next version of the laptop is incremental, very important, but something that you need to do. At Ansys, Ansys Mechanical is a finite element based structural solver. We are doing the next version, it's faster, it's a little better convergence, better meshing, but it's still the same tool. So that's what large companies do. Academia, we invent new things. We are doing disruptive innovation which I call Horizon 3 innovation. The truly disruptive innovation always happens in startups. Large companies actually struggle with disruptive innovation. In fact, Neil, I have done broadcasts on this, I wrote a book called 'The Innovation Factory' which your readers may be interested in, and the whole premise of this book is how does a large company like ABB or Schneider or HP or Ansys, the companies that I have actually worked in the role of CTO, how do these companies try to foster Horizon 3 disruptive innovation? Large companies doing disruptive innovation is a concept called open innovation, and that is what I am truly passionate about. So I know you asked me a question about the difference between academia and the large world. Academia does discovery of knowledge, Horizon 3, but they don't actually make products. This disruptive innovation, there are people like me who leave academia and they build the disruptive thing but in a concept of a startup. A startup is laser focused on that one product the world has not seen, very disruptive, but that's the only thing that they do. So they're focused on it, and then they'll do the second product and the third product, ultimately that will also become a large company at which point it will stop doing Horizon 3 innovation.

The way I would think about a startup is, if you are doing a startup just to make money, you've got the wrong motivation. The motivation is really you are trying to solve a problem that the world has, and you see no solution. There's no existing solution from the large companies. You're trying to do this fantastic computer that will... They see a problem and they say, 'You know what, I'm going to solve this.' And typically in a large company, the manager will allow you to only work on things that are incremental. So as I said, you are working in HP, you're making laptops. If you say to HP, 'I want to build a quantum computer,' your manager says, 'Go away, that's not what we do.' But often times these problems come out and look at you and say this needs to be solved, and you have this burning passion to solve that problem. And sometimes your company manager will allow you to do it, then you're lucky, then the company is actually allowing you to do Horizon 3 innovation. But 90% of the time you will... The other way is for academia, academic people. So literally, startups come from two ends. Either it's an academic who has solved a really hard problem and says, 'Okay, now I want to commercialize it,' like me. And again, I am just a very small person, but there's so many more famous people who came from academia of absolutely wonderful companies, and I mentioned them in my book. And then there's startups that happen from... I would say 80% of the startup founders actually come from the large corporate world. They have found a problem, solved it, and then they start one company, they start a second company. Now, you asked a question, what is the ultimate? Yes, so you get motivated by solving the world's problem, and then how do you establish the market? That is the hardest thing for a startup entrepreneur to do. So often times you say, 'Well, what's the market for GPUs?' You can take a look at Nvidia and AMD and so on, and okay, these are the people making GPUs, they're selling this many GPUs, so the market for GPUs is this. And if you are a new startup trying to do another GPU, you know exactly what that market is. What's the market for eyeglasses? I have eyeglasses, you look at all the people who are wearing eyeglasses, you can say that. But suppose you are a startup, you are inventing a device such that blind men can see. What is the market for that? The market is zero, therefore it's a bad idea, I should not do it. Because those marketing things done by companies like Gartner or Dataquest, they are only looking at markets where products exist. What's the market for the cloud? It is 100 billion. The market for cloud before Jeff Bezos invented AWS was zero. But it took a person of Jeff's imagination to say the market is this if I could build it. So for that device that blind men can see, I'm the entrepreneur, I'm trying to find the market. I say, 'Well, how many blind men are there in the world?' I know I have 10 billion people on the planet, I don't know, maybe 3 million blind people. But if I can improve their vision, at least I'll pay 200, maybe 300. So 300 times 100 million blind people, that's a 3 billion market. That's how you size the market. So you have a choice of making a device such that blind men can see, the market is 3 billion, versus a chair with nine legs, and the market for that is only $2. You should pick the first one, even though that is a harder problem to work on, because if you are successful you will solve a world's problem and it's a large problem, versus inventing a chair with nine legs which is a simple thing because you know a chair with four legs, it is easy to do with nine legs, but the market is only $2.

It's a great question. So let me explain my thought just by going into the area of simulation itself. So I want to... The world around us is governed by the laws of physics, which are second order partial differential equations. So you can write those PDEs, and when you went to college you can take a very simple differential equation, linear, the simplest one you could analytically solve is e to the minus 2, something. This is how the equations go. But in the practical world, these problems have such complicated geometries by the time you take the CAD, define the boundary conditions, and so on, and you have the Navier-Stokes equations to solve, it is impossible to solve it analytically. So you have to solve it numerically. So you take those PDEs and you discretize them. You do finite elements, you have boundary conditions of the other nodes that are next to you, and you keep iterating on it, and that's how all our numerical methods work. The trouble with these numerical methods is the tradeoff between accuracy and speed. So suppose you solve that problem, the CFD, with a thousand elements, and you get an accuracy which is about 10% error, which may be fine for you, and you solve that in an hour. Say I don't like 10% error, I want it to be more accurate. It is very easy in our world to just instead of a thousand elements do 100,000 elements, and you get 1% error but it takes 100 hours. So the speed in our world of CAE simulation, CFD, is this problem: the accuracy versus speed. And we want both, we want both accuracy and speed. And furthermore, the third thing is these things are so complicated in terms of convergence, sometimes you do these crazy things with the machine and it doesn't converge. 'Oh my God, I didn't converge because of this, I should use a different mesh, I should use a different solver.' So there are these zillion tools that I have at my disposal, and the CAE analyst is using all of these things, and sometimes it works, sometimes it doesn't. So it's not that easy to... From a user's perspective, they just want to press a button and it just does it. That's the ultimate holy grail. So in our world, the problem is you have to be accurate, you have to be fast, it has to be easy to use, and converge all the time. That's the holy grail. So in my role as CTO, I look at all the solvers, I say how can I get to that current state to make it more accurate, faster, easy to use, and so on. So I have one pillar on numerical methods, and we just with advanced numerical methods, without using high performance computing, without using AI, you make it faster, easy to use, converge all the time, and so on. The second pillar is HPC. You work at AWS and you have all those high performance computing. So we take an algorithm and we parallelize it, put it on 100 processors using shared memory or message passing with distributed data decomposition, or with GPUs. So there are all these different things, but this is what I call brute force acceleration. I have a job that I've decided that I will use 1 million elements because of the accuracy I have, and it's taking me 1,000 seconds. I can use 1,000 processors and make it run in 1 second. That's brute force parallelism. The third pillar that we have is AI/ML, which is sort of your question. So AI/ML has been used in a variety of fields, but it has been used for recommendation engines, 'which restaurant should I go to?' It's wonderful for those things, or ChatGPT allowing you to write wonderful poetry and text. But the question that we asked is, can AI/ML be applied to numerical method simulation? And that's when I joined the company six years ago. My CEO said, 'What do you want to work on?' I said, 'I want to work on AI.' And the early work on AI that we did was to say, 'Okay, let's take a...' You have an AI model, and you have this six-stage neural network, and you have the weights of the neural network. You don't know what the weights are, so you start with some random weights on the neurons. You have an input, here is the output. So with random weights you will predict an output which will be completely wrong. There is an error at the output. You say, 'Now that there's an error, how do I minimize the error?' I do back propagation to adjust the weights of the neural networks so that my error is zero for this input-output combination. Then I give it a second input with a different boundary condition, different whatever, and with the first two sets of weights and my third input, I keep iterating. After about 100, 200 cases, I kind of converge on the set of weights on the neural network. And within that, there's all kinds of choices: should I have a six-stage network, should I have an eight-stage network, how many, what's the depth, what's the width, and that ties to the parameter size of your network. But assuming you have done all that, that's what SimAI does. So SimAI is a platform which allows a customer to take their... You have your problem, and then train it for the first 100 designs that you have, and the 101st design, instead of taking 100 hours, will run in a minute. That's the value proposition. Now, the AI is only as good as the data you train it with. So if you train it with this picture of an SUV, you train it with this SUV from Toyota, the 10 different versions, the Highlander, the 4Runner, the RAV4, and also the SUVs from Hyundai and the SUVs from Ford, so you are training it with SUVs. It learns. Then you give it a... If you are Airbus, you're making airplanes, you are not going to go from one airplane to tomorrow doing a submarine. You are actually doing only airplanes. So it is actually worth it. There is value in subtle variations, and that's what these designers do. They're doing thousands of designs of slightly different airplanes or slightly different cars, and so on. So there is value in SimAI. But then you ask a question, where is the future? The future is foundational models for AI, where the customer will not have to train any set of things. There's no need for a... ChatGPT is a foundational model. It has been trained on all the words in the English language and has learned how to speak, how to write poetry. The grand vision of AI with foundational models for physics is to do that. It is an incredibly hard problem, but that's what we are working on.

Yes, and here's why. I'll go back to the Google example. Google search is so good because it's a free tool. You and I type things on Google, and based on it, they are creating this massive database, this PageRank algorithm, and based on me clicking this, it gives you the list. If only 10 people in the world used Google, the search would not be as good. The reason Google is good is because they are looking at the 10 billion people on the planet banging on our keyboards for free. We think it is a free thing, they're not paying us to give them the data. They are using all our information to make the search better. So in exchange for us getting a free tool, we are giving Google back the knowledge in our head that after I type 'Who is Neil Aston from AWS?', somebody in the world is actually interested in the question of who is Neil from AWS, and the other question is who is... Think of it as the Google search for only people within Airbus typing the searches, versus letting the searches go to all engineering companies, to Airbus and Boeing and Pratt & Whitney and GE. It will be clearly richer. That's the value. Now, to make it happen, I anticipate the question: where do you get the data from? This is something I am actually thinking of. To build these foundational models, I will have to get all the CAD files from Airbus and all the CAD files from Boeing. But they will say, 'Why should I give you my data?' So I have to create a model which is an open model where all Ansys customers would opt to give their data in an anonymized way to Ansys to collect the data, run all those things, for example on the AWS Cloud. And the cloud is a great way to train all these models because if it is on-prem, you actually cannot have access to it, but if you are going on the cloud, if every customer, if all CAD designs are down on the cloud, it is actually possible. And so if that were to happen, it would be more accurate than the Airbus-specific result. That is the long answer to that question.

I am an artist, I'm whatever. Prithviraj Banerjee has written a beautiful picture, and you took that picture into DALL-E and now you generate a new picture based on its Indian knowledge. That's not fair. If Prithviraj Banerjee were to say, 'I will give 10 of my pictures to DALL-E or 10 of my poems to OpenAI, and I get one cent for everything that I give, every token I give to contribute to this thing, I get one cent,' hey, I am incentivized in that case. I will not sue DALL-E. So my thing is, I think the whole world of governance mechanisms, people getting sued because they... They have to figure out how to take that big thing into small chunks and to figure out a royalty of one cent per pixel. I think the world will actually go in that area. I think the world will really figure out governance and fairness so that everybody wins.

That is a great question. Actually, the answer is there's not enough research that has happened. We need to get to the right level of accuracy for the models where physics-informed data... So I'll give a very simple explanation. Suppose you're trying to look at fluids data and trying to put it into an AI model for fluids. You will take the fluids data: here is the velocity, pressure, temperature, and so on, and this is the distribution. And you think the whole thing is random, it is not, because the fluids physics says there is the Navier-Stokes equation, there is energy conservation, all the stuff that you know from a physics point of view. So the data will not be completely uncorrelated. The data is... An independent variable, that's the property of statistics. You think not all the things are going to be independent. So the pure data-driven approach assumes everything is independent, and it's not. So if you can insert the knowledge of the physics, you can constrain it. You don't have to search for millions of data points, you can do it with only a thousand data points. That's the power of physics-informed. And the work was, as you know, done by George Karniadakis at Brown University, and we did a lot of work at Ansys, we've done a lot of work at Nvidia on these things. The trouble is, when we started doing the physics-informed to incorporate the physics, the computation needed in this... The models are very expensive. Once that thing is invented, it's like the Einstein theory of relativity, the universal thing. Something like this will happen where we'll merge the areas of numerical methods and AI. And that I have told my board is when the whole market for Ansys will completely collapse, because for the last 50 years we have worked on the fact that it's all numerical methods and so on. Numerical methods will no longer be needed, it will all be done with AI with the accuracy and the speed much much better than numerical methods. But we are not there yet. That's where the research is.

I think the reason the AI world has worked so fast is because of open source. You have things like TensorFlow and PyTorch, these are absolutely open source ways of doing neural networks. Google and Facebook could have kept all of those completely closed, and then the world wouldn't have done all this kind of stuff. Nvidia opened up CUDA, so CUDA became sort of... the code is not open, but they have an open framework. So the combination of open source things like CUDA, open source things like... In the numerical methods world, we know something, Dassault knows something, Siemens knows something, and we don't sort of share stuff. So there's a paper that'll come from CMU or Stanford, some wonderful people, and we say, 'Ah, but they are working on problems they cannot work on, but Ansys will not give those top problems.' That did not happen in the AI world. That did not happen in the MapReduce world. In the MapReduce world, the MapReduce thing was actually openly given away open source by both Google and Yahoo. Now, why did they do that? The MapReduce framework is a framework that Google... So all the innovations coming from the open source world, academic world, Google could put in and make the search even better. They did not say, 'Here is a search algorithm that we open source.' They took a core part of their search algorithm which they are making money from with ads. So I thought that was an absolutely brilliant strategy. Linux is another brilliant strategy for advancing operating systems. So we have to actually learn. In our world of CAE simulation, there is obviously one code called OpenFOAM, since you know fluids. We need in this area some work on 3D geometries of all kinds of things on gears and propellers and airplanes and so on. If you can do that, I think that will advance the state of the art. And we ourselves published a couple of datasets, these are DriveML and AME-ML, which have helped a little bit, but not at the level of ImageNet.

You are absolutely right. With large language models for words as tokens, they have scraped all the words from all the books that people have written. Of course, they are not giving the royalty back to the people who have written those things. They have not generated those. Suppose the next version of ChatGPT is to take ChatGPT to generate all those tokens and then you feed it, that would be your problem. And DALL-E has taken the same approach of taking all the images on the internet and using those images to program DALL-E. And now they have videos. So imagine you sensorize your car to measure the fluid flow at every small millimeter of your car. That is an actual measurement of how the air flow actually happened on the car. But you have to take thousands of cars, millions of cars, it's just ridiculous. So what I have told my board is, Ansys will create the synthetic data through simulation of all the fluids models. Absolutely. Ansys Fluent is a RANS simulation, so it will not be generating the LES, so we'll also have to do the LES. In the LLM world, it is a trillion parameters. In our world, it is probably a billion trillion parameters, I don't even know what the size of the model is. But it is possible, I absolutely think it is possible, and we will eventually get there.

Let me make this observation. Isaac Newton made a bunch of observations in the real world. The data that he fed into his engine determined the law which is force equals mass times acceleration, which is a differential equation. He deduced the law of gravity by observations. It is therefore possible to observe the world around us and actually... We have got work going on with Google DeepMind. DeepMind is using the Ansys tools to observe the physics and learn the physics. Imagine you... The world of structures, and just by... They have trained the robot arm to do the balancing of this by learning the physics. So it is possible, and that is how I think foundational models will work. Because Isaac Newton generated the physics model of force equals mass times acceleration by looking at the data, by observing the data. AI is going to observe the physics around us and train the physics models. Every one of those equations can be deduced. Navier-Stokes equations can be reverse engineered by AI.

It's a great question. Actually, in my CTO office, the first thing I did was to work on AI. And now that the AI thing is sort of not solved but at least we have some products out in this area, we have started working on quantum for exactly that reason. And the beauty of quantum is there's a potential for exponential speedups. Because if you have n qubits, your runtime speedup is 2 to the power of n. Quantum computing algorithms have been used on problems that are sort of NP-complete...