Well, thank you so much for the introduction. It's such a pleasure to be here. So, what a nice spot to be hanging out. And I've had a very cheese-intensive morning, so I feel like I'm experiencing Wisconsin and hopefully can make it to the end of this talk.
But at any event, it's such a pleasure to be here today. I tried to tailor a little bit to this audience and we'll see if I'm successful or not. But mostly just to share some research from our team at MIT, specifically aligned a bit to this generative modeling space with somewhat of a geometry flavor. So, I'm around all day today. It's my pleasure to chat over coffee with just about anybody while I'm here. For those of you who I've not had the pleasure of meeting yet,
ah, okay. There's like Zoom layered on top of other stuff and switching slides appears to be a challenge, but we'll make it work. In case I haven't had the pleasure to meet you, I'm Justin. I come from MIT. I lead a research group with a purposefully ambiguous name, that we study geometric data processing, and I'll say a little bit more about that in a second.
MIT is, I think, one of the most wildly bureaucratic and confusing academic institutions. Because of that, we're attached to all kinds of different efforts within the university, but essentially this is a research team in applied geometry that studies problems in all kinds of different spaces, from statistics to computer graphics and numerics.
So, specifically our team has kind of a funny, like Bert and Ernie kind of situation between two different areas of geometry that I think are often considered very different, kind of distinct parts of computer science, but largely as a byproduct of my own weird PhD, which I started in one and ended up in the other, we have some research in both.
So, on the one hand, our team works a lot on problems of processing geometric data. So, for instance, my own personal background before I started a PhD was working in an animation studio and in TV. I have a lot of friends in the computer graphics world, where really the algorithms that you're trying to work with are what I think we would conventionally think of as applications of geometry in computer science, things like physical simulation, autonomous driving was one of the big ones that we worked on the last maybe 10 years or so, where for example the kinds of problems that we're thinking about involve like, how do you design a machine learning model that just looks at a piece of geometry and makes an inference about it? Or how do you design algorithms for simulation, 3D modeling, and so on.
But then the other half of our research team works on kind of geometric approaches to processing data, where now your data is maybe embedded in some high-dimensional domain, and essentially our sort of ongoing thesis is that these two areas are very strongly supportive of one another. I have a lot of students that cross the divide from one side to the other and back and are able to think about using these nice low-dimensional problems that are very tangible to inform the reasoning about kind of abstract high-dimensional machine learning problems and vice versa.
So, for example, my own PhD was in an area of maybe statistics and optimization known as optimal transport, which at the time we thought was going to be a really cool tool for keyframe computer animation. And in retrospect, that was a really bad idea because the whole point of optimal transport is to be as uninteresting as possible, which of course is not what computer animators want. But it did end up being a very useful statistical tool in some inference settings. And so sort of the structure of our research team follows that weird history from my own work.
So, today we'll be largely in the second category. Though hopefully that's aligned with the seminar. Of course, at any point if you want to derail, it's very easy to do that, and I'm more than happy to tell you about all the fun problems that we study in the computer graphics domain as well, largely with PDE and optimization.
Before I get started, I should tell you of course that I'm just a conduit for all the cool stuff that my students are doing. Really, if you have hard questions, these are the people you should ask. I'm here to just, you know, take the credit and do what professors do, I guess.
But really, I'm so fortunate to work with this team of really brilliant folks at MIT, and really my job is to just stay out of the way. In particular, I wanted to highlight that most of the work I'll tell you about today — I've tried to be careful with attribution and all that — is from my student Chris, who's just about to graduate and is starting a faculty position at Rutgers. So, if you like this kind of stuff and you happen to be an undergraduate or a master's student here, I would strongly suggest applying to Chris's research team because it's just getting off the ground and he's really awesome.
Okay. So, in any event, we'll dive into some technical stuff with a little bit of high-level motivation, and then, time permitting, I'll try to tell you about three different projects we've had in our team, roughly at this kind of intersection of geometry and machine learning.
I guess that's kind of a reasonable high-level starting point. Here's kind of a funny thing that I think it took me a long time to realize. I mean, our research team is certainly on the fringes of the machine learning space. I don't think we can claim to be sort of central researchers for many, but certainly we interact with these tools daily.
And one of the things that I found so overwhelming when you look into the machine learning literature is you look up all these different methodologies and architectures and so on for learning. Here I just went onto Google Images and looked for, you know, a diagram of a machine learning architecture, and you see these wild pictures with circles and arrows and human brains and weird robots and cats.
But the reality is that I think for a lot of the analysis and statistical problems that we study, even going back to just the multilayer perceptron, we can abstract away a lot of these different architectural details and think of a machine learning architecture in this vastly oversimplified fashion as a sequence of matrix multiplies and nonlinearities that take you from the sort of raw data format that's flowing into your learning model all the way to some decision.
And there's all of this mythology and understanding that has to go with what that sequence of transformations looks like, right? And especially these days in this world of large models where there's these matrices of millions and billions of elements and there's so much going on, I think it's a pretty reasonable kind of analogy to think through how these different layers of your machine learning architectures are taking the geometry of your raw data set and then slowly and progressively reshaping it into some format where you can do inference or whatever your task is in a more abstract fashion.
So from that perspective, of course, your learnable parameters here are maybe elements of these old matrices, and you get this sequence of embeddings as the outputs of your layers. Now, obviously, those of you who are machine learning engineers will be very quick to correct me and tell me all the different reasons why this is a really bad abstraction, and I totally agree with you. But I do think from just a high-level perspective, that'll be good enough for us today.
And indeed, this perspective has been around a long time. So, for example, here's sort of a famous early experiment that people did in convolutional neural networks, where one thing that you can figure out is that with many of the state-of-the-art trained CNNs, if you look at the sequence of embeddings, what they're essentially doing is learning some version of abstraction of your data that goes from basically some wavelet-looking feature all the way to high-level stuff that's trying, in this case, to represent a face.
Incidentally, I'll take these like 18 seconds to argue that I find these diagrams to be completely worthless, and just the world's most complicated way to illustrate literally a matrix multiply, right? That's all these pictures are. And I wish I knew that when I was in undergrad.
In any event, so because of this sort of very abstract picture, I think there's a lot of really interesting possibilities for research that I think in some sense sidestep one of the big concerns that we have today in modern machine learning work, which is that a small lab full of math dorks like my own just doesn't have the resources that we need to be able to train some giant model.
We just don't have the compute, we don't have the expertise in order to, for example, make our own whatever the latest buzzword is, like a foundation model for processing data. But what we can do is take all of these different models that exist out there in the world, and try to take a pair of tweezers and understand how they behave, how they transform our data a little bit from a phenomenological perspective, and also from a mathematical one, and then try to leverage that and reshape it in different ways that are useful for other tasks.
And so, for example, this gives us many different embeddings of our data sets that give it different geometries. For instance, I put in quotation marks here because I'm a geometry person. You know, people talk a lot about there being some manifold underlying your data set. One might ask whether that's really true or not, but it's certainly a reasonable analogy.
We can talk about how to adjust the geometry of these embeddings to make them better for different tasks, or even how to maybe regularize some small-scale model to have some behavior that imitates the larger case.
So, this is obviously a very abstract and high-level story. I mean, really my goal here is, in a very loose way, to glue together three research projects that I thought were fun and I'd share with this audience here. I guess this is my luxury post-tenure, is to be able to just say that.
But essentially, I think all of these can be understood as different techniques for understanding the shapes of embeddings that kind of come out of different layers of learned models and manipulating and navigating those embeddings in different fashion.
So, in particular, I'll start by telling you about some of our work in the diffusion model space. Diffusion models these days are some of the trendier models for generative AI style problems. Specifically, I think image generation is the main application where we see these things getting used, but these days also you see them in robotics, you see them in audio, all kinds of different domains with varying levels of success.
And I think one of the questions you might ask is kind of like, what are the statistics of those domains that make these diffusion models work so well in these different settings? I'll tell you a bit about that.
Then I'm going to switch mode a little bit and tell you about some of the space that we have in fine-tuning and adjusting learned embeddings. So, this first one is all about just navigating. Like, once you've learned a useful embedding of your data, now you want to start generating new data points that kind of are distributed like the old one. A different thing you might want to do is take your favorite language model or statistical model and adjust it to be good at some particular task or maybe to have knowledge of a data set that I cannot use to train my full large model. And we'll talk about some interesting work that we've had in that domain as well.
And then finally in this third thing — actually I probably should have swapped two and three because one and three are linked to each other — I'll tell you about some work that we have involving trying to design smaller models whose intermediate layers have some structure in common with these large learned foundation models that aren't regularized explicitly. So that's what we're planning for today. And if at any point you're bored or you've had enough, just let me know and we'll call it.
So with that, let's get started, and in particular I'll tell you about some of our work in diffusion modeling. Again, I should be careful to note this is largely work that's been led by my student Chris, where really this is the byproduct of his vision in this domain.
And I think really stands out as a bit of an interesting combination of perspective, not just from statistics but also in some sense from just computer engineering and even numerical methods, which was good for a person like me that has more background in those spaces than in statistics.
So we'll start out with the sort of basic setup in case you're not familiar. So in general in this kind of generative learning space we have this somewhat ill-defined task, right? We're given a big data set of data points. So think about like a big collection of photographs that are collected from the internet, like every cat on Google.
Now each of those data points is some x_i, and our typical mental model is that those points are drawn from some underlying distribution, right? So for example people talk about maybe the distribution of natural images on the internet. And obviously we do not have access to that distribution, but we do have samples. Right? In other words we have individual photographs, but we don't have like the density function of that object. In fact it's not even obvious that a density function exists.
And then our task is to now draw novel samples. And I want to pause here because I think this is typically how people motivate this kind of generative machine learning task. And I want to note that this is a totally ill-defined thing, right? Like it's really not obvious. Like in some sense, what does novel mean? And of course without having some prior understanding of our distribution ρ₁, then this task really makes no sense at all.
And so it's one of the kind of miracles of modern generative machine learning is how far we've gotten without even kind of writing an explicit prior on these things. And oftentimes we're really relying on our architecture, our training procedure, and so on to provide some species of regularization that a posteriori we try to explain, which is very different than, for example, the world that I grew up in where you carefully design maybe a Bayesian prior or something like that, you analyze the bejesus out of it, and then you try it.
So we're in this new kind of universe where it's kind of experiment first and write equations later. And so our team is going to try and bridge that gap just a little bit.
So in particular within this domain, there's a class of models that are particularly popular. This includes diffusion models, this is flow matching. I'm sorry, Zoom has 12 different reminders on my screen at once. Okay. Which do the following thing.
And I think really in some sense is reflective of a high-level breakthrough in this kind of generative or sample-based approach to machine learning, which is rather than just in one shot trying to learn how to generate samples from a distribution, instead you kind of learn in a progressive fashion. And this is true both for images and for text, right? So for example, these autoregressive text models are somehow doing something similar.
But in diffusion models, essentially what you do is you define some stochastic process where you start with a distribution you know how to sample from, like a Gaussian.
And then you write down maybe a differential equation, either ordinary or stochastic, that takes you from that guy to the actual distribution you want, like this checkerboard here. And the learnable parameters are maybe the dynamics of that process, right? And so in particular, your algorithm looks like, you know, if I want to draw something from our unknown distribution ρ₁, I draw something from ρ₀, which is usually just some boring Gaussian thing. And then I just solve like an ordinary differential equation that flows through some velocity field, and that becomes our sample, right?
Or there's like a slight variation of this thing where you basically do some version of forward Euler integration and add a small Gaussian, and then you get to use fancy SDE language instead. But that's really all that's going on behind the scenes.
And there's all kinds of different ways that one can go about this. I mean, one thing that you might know from either fluid dynamics or undergrad ODE or whatever is that this velocity field is not unique. There are many different ways to take a Gaussian and map it onto the same distribution here. So there's actually a bit of a design decision, but in some sense that's where engineering happens, right? Some of these are probably easier to learn than others.
Okay, so specifically, a class of algorithms that seems to be very effective includes diffusion models and flow matching. These are basically the same up to some rescaling of variables, which learn the velocity field in a really interesting and straightforward fashion.
Essentially, remember that V here is the velocity field of our dynamics. This is like the thing that's moving samples from a Gaussian to your final distribution. And in particular, you set up this simple variational problem, which does the following thing. Sorry for some math notation here. I can't help it. It's a professional hazard. It randomly draws a data point x from my data set, that's ρ₁. I randomly draw an ε, which is just normally distributed.
Notice that basically I can draw both of these things. What I don't have is the density for ρ₁, right? But I can go out on the internet and download a cat, that's easy enough.
And now I have this particular loss function, which essentially is trying to do the following thing, where the velocity field is kind of looking like the difference between the Gaussian random sample and the sample from your data set. And one thing that you can prove is that if you optimize this thing with respect to functions — not neural networks yet, just functions V — then the velocity field V here truly does take samples that start at a Gaussian and transform them to your final distribution. This is called flow matching, right?
Okay. And by the way, that's not entirely obvious. You have to actually justify this thing. I think the easiest proof kind of comes from a nice application of Bayes' rule, but this is a very straightforward procedure and one that looks a lot like how people train diffusion models today, which is very simple. You randomly draw data points, you randomly draw Gaussians, you interpolate, then you try to get your neural network to predict that. Okay.
So, this is a very elegant and simple training procedure. And indeed it is roughly what people do in practice. And initially what was proposed, including in this ICLR paper and some earlier ones as well, is that you parameterize V as this unknown neural network. This thing becomes your loss function, where you just randomly generate or draw these things and then you can do stochastic gradient descent. Hopefully that story makes some sense.
Okay. But here's kind of an interesting thing. Let's think about this as an optimization problem, not for a neural network, but for a function, for a function V. Right? So before I begin inserting all the complexities of like neurons and activations and so on, maybe I'm a mathematician and I want to say like, what is the optimal solution to this problem in an ideal universe where I can just find any function V I want. I'm not restricted to the set of things that my neural network knows how to learn. Hopefully you guys understand the distinction there.
And I think, you know, people often throw around like these universal approximation theorems as some bridge between these two, right? That like if I make a really big neural network, then essentially I can approximate any V. So, I think that's a reasonable question.
And here's the thing that I think is kind of interesting about flow matching and other problems in this domain. As a function of neural network parameters, this is a very complex loss function because of course it is. There's ReLUs and matrices and all kinds of stuff. But as a function of V, this is least squares. Right? And here's the thing you might remember from undergrad linear algebra class, least squares problems have solutions that you can just write down. There's just a closed-form formula.
And in fact that actually is true for diffusion models. There's this kind of interesting, I would argue slightly neglected fact, which is that if you truly believe the story on this slide and you truly believe that our goal in life is to make our neural networks as large as possible, then in some sense we know what these are going to approach and we don't need a neural network at all. We can just write down a formula for V.
I mean, I decided to write this squared function which turns out is linearly related to V. I don't know why I did that, but essentially the formula for V looks very similar to what I wrote on this slide. But it is really boring. It's just a linear combination of your data points.
And that kind of makes sense because let's think back to our original story. Remember our goal here, I have a bunch of points sampled from a data set and now I'm trying to make a sampler that kind of imitates that. What's probably the best sampler in the universe? Well, it's the one that randomly draws an index in your data set and gives you back that image.
And in fact that is the global optimizer of this objective function here. If you look at the globally optimal V, it's the world's most complicated way to sample from your data set.
And so my student Chris asked a totally reasonable question, which actually I think was a little bit surprising to us was not studied in this domain, which is: you have a closed-form solution for the diffusion model problem as you wrote it down. You have the thing that comes out of neural networks, and the sort of numerical analysis brain of mine would say, 'Well, okay, well then we should compare these two and do error analysis and try to understand like is the procedure that people use for these generative models at all resembling the solution to these problems.'
And of course the answer is no, because if it were, well, then you would have a bad sampler, right? In some sense the whole reason that these diffusion models are performing well is because they do poorly at the task at hand. Right?
And so I wanted to pause here because I think this is a really interesting kind of philosophical question. In fact, there's a few that are all tangled together here. Why do we use neural networks for this problem? If we have a solution to it, it's just a kernel, right? It's a Gaussian kernel at the end of the day. It's called diffusion models, but it's not surprising that this, you know, at least for small data sets, it's not obvious that I need to train anything at all.
And so there's a lot of hypotheses you might guess for like why we might want to use a neural network for this. One is that it's compression, right? So in particular, if you look back at this formula that I wrote down here, there's a giant sum over your whole data set, and maybe that's intractable. So I'm using my neural network to just kind of remember my data.
Now that's a testable hypothesis. We can just at least for a small enough data set compare those two and see. I think the answer is no. So we don't think that's likely. But the second is probably that well, at least we get the right vibes out of these models, you know? They generate interesting samples, they feel novel.
As we know, neural network training, due to some combination of probably stopping before optimality and the architecture itself, leads to some bias where the samples that you get are not just in the training set, but there are some that are kind of nearby and distributed similarly, right? And so most likely the second one is kind of the explanation for why in practice we use a neural network for this problem.
Which of course begs the question, which by the way I'm not going to answer. I'm asking this as if like the next slide is going to give you an answer, and sadly that's not the case. What does it even mean to generalize for these kinds of tasks?
Because in particular, I think we're all stuck, you know, at least for me, I teach our undergrad machine learning course, and we teach this beautiful story about generalization for supervised machine learning, right? That you have a data set, you withhold part of your data set, you train on this one, test, you have these nice generalization bounds that kind of say, 'Well, you know, as long as my model is expressive but not too expressive, then maybe I can make good predictions on data I haven't seen yet.' The phrase 'data I haven't seen yet' actually really doesn't make sense for generative modeling. There is no notion there.
And so a lot of I think what you really see in this domain is just qualitative in nature, right? You look at the images of the models and you say like, 'Yeah, do they look kind of novel but not too novel?' And that's as good as we can do. And so it's kind of an interesting space where there's a lot of math. I mean, when you read these papers, there's like stochastic differential equations and all kinds of crazy bounds getting thrown around. But the actual thing that we're after is quite tricky. And I pose that more largely as a question because I think it's one that's worth thinking about. Like, how do we make this systematic and thoughtful? I think it is really an open thing.
So, in any event, one question you can ask just from a modeling perspective is you could say, 'Well, okay, so my neural network for learning a score function or this velocity field V is not imitating the global optimal solution. So, one thing I can do is kind of work around the edges in both directions. I could take the learning procedure and edit it, or I could take my closed-form formula and propose small changes and see if I could introduce some kind of bias to the closed-form model that maybe imitates the reality of neural network training a little bit better.'
And indeed, for fun, we played with a few very simple things. So, one of the hypotheses here is that neural networks have a bias for learning overly smooth functions, right? This is a well-known thing. And so, one thing you might do is just try to kind of smooth out the learned velocity field that comes from this procedure. You have to be a little bit careful how to do your smoothing, it turns out, because some kinds don't make sense for this particular sampling problem.
But what we found is by basically convolving the right kernel against the Gaussian inside of that formula, at least for very simple sampling tasks, we can overcome a bit of this 'just sampling from your data set' aspect, and actually imitate, at least on small data sets, the same kind of bias that you see in learned diffusion models at the larger scale.
Now, what I don't want to claim is that we have a state-of-the-art diffusion model with a formula with just inserting a Gaussian in the right place. That's awfully optimistic. But what we can do is begin to model the sort of things that get learned in this procedure, and I think really we have more questions than answers here.
But essentially, I think there's a world where you say, 'Okay, for all we know, 95% of the training time for a diffusion model, which is usually quite large, is spent just regressing a Gaussian, and then the other 5% is spent learning an interesting embedding of my data set, right? Then that would be a good thing to know, because then maybe I can make a really efficient diffusion model that bypasses all that boring stuff and just learns the interesting part.' And sort of our initial experiments in this domain seem to suggest some possibility that that could be the case.
So, for example, maybe I'll skip over that. We have a very simple smoothed-out formula that, after a lot of tuning of exactly the right kind of smoothing, we can get to, at least on very simple examples — for example this goofy data set which everybody uses as kind of just a sanity check — begins to kind of have the right bias for these sampling tasks.
And then we were able to get a sampler for images, not on the largest data set, but at least on this kind of CelebA, you know, faces kind of data set, a relatively good sampler by combining basically two things. One is smoothing out the closed-form formula, and the other is really working very hard to learn a good embedding space for your data set.
And so I think this is an interesting kind of question to ask, which is we put a lot of statistical effort into learning a good embedding of our data into some space where it probably forms a much simpler shape than where you started. And I'll tell you more about that in project number three. I'm realizing my talk is organized in a poor fashion, and for that I apologize.
But after we do that, we can do sampling in that latent space using this very simple closed-form procedure, no machine learning involved, and actually get pretty convincing output for our sampler.
Now, this is clearly not on par with what you're seeing from Google or OpenAI, but I do think it's a really interesting kind of little motivator for asking these questions of like, exactly what are these models learning, and can we distill our training procedure to just learn that one part, and maybe bypass learning all the other stuff, which is just a mathematical formula like the world's most complicated Gaussian. And I think we just don't know. And there's some really interesting scientific questions here.
I should share we have a few follow-ups for this paper. Sadly, I'm already over time, so I won't share too many details today. But my student Arpit looks specifically at the image model case and found in that particular domain a really surprising result, which is that these trained models actually resemble just a linear diffusion model extremely well for like 95% of the sampling procedure.
So in other words, like this velocity field V, specifically if you assume that your data set is Gaussian, then it becomes a thing called a Wiener filter, which is classically known in image processing, and you can replace many of the diffusion steps with that thing and not affect the quality a whole lot, which is like another clue that maybe there's some simpler structure going on there.
And as a byproduct of his work, he sort of — I don't want to say disproves, but gave some evidence against an earlier hypothesis, which is that maybe the reason that these diffusion models generalize has to do with the machine learning architecture. So, for example, for images, maybe you're using a convolutional neural network where there's some translational invariance, which isn't in that original problem that I wrote down. But what he showed is that the same phenomena emerge even when you don't use architectures like that. That it has more to do with the statistics of your data probably than about the architectural decision.
And then most recently, my student Chris again did a really clever calculation, which I like. This is related to influence functions in statistics. I know some people here work in that particular domain.
Which is, if you're looking to do all kinds of fun mathematical analysis of diffusion models, the fact that it has a closed-form solution makes doing your math a heck of a lot easier because there's just a formula. You can do high school calculus on that instead of having to prove complicated byproducts of theorems about differential equations.
And so one of the things that you can do is just differentiate that formula specifically in the shape of your data set. So in some corners of statistics, this is called computing an influence function, at least if it's on the weights of your data set. And what that gives you is a really interesting thing, which is a sensitivity analysis.
So one question that you might ask is, let's say that I introduced or removed points from my data set, how does the output of my generative model change? So for example, maybe I want to know like, oh no, like this thing is creating copyrighted content, you know, what part of my data set is causing that? Remove it.
Then you might want to ask questions like this, or like I think another one that is neglected and is worth studying a lot is attribution, right? That a lot of the data sets, especially for creating images, are curated from artist-created content. And one of the big ethical discussions that we should be having right now is whether it makes sense to generate new images and not credit those artists or give them compensation.
But the first step in that direction would be simply to figure out like what pastiche is being created by this generative model, which is currently unknown, right? But when we evaluate these black boxes, it's really hard to go back and attribute a given generation or utterance from this model to a particular data point. One way to do that would be to solve a sensitivity analysis question, like how much would my output have changed were I to have removed this piece from my data set?
In any event, because we have these nice closed-form formulas, what happens in this research paper, which is currently on arXiv, is you can work out these sensitivity analyses in closed form. You can just work out this derivative pretty cleanly. And the kind of nice thing is you can write it as a recurrence. You can write it in terms of the diffusion model itself.
So, Chris did a really sneaky thing, which is, okay, you use your closed-form formula to derive your sensitivity analysis, but you actually can write it in terms of just evaluating the model multiple times, which allows you to do it on trained models, assuming there's a bit of a leap of faith that the same sensitivity formula should still work even if your large model is kind of inexact.
And so, this actually gives a procedure for analyzing not just models that are done in this very straightforward closed-form fashion, but also the learned ones as well. And we show empirically that that's actually a pretty reasonable engineering decision. So, more to come soon.
And then finally, my student Artem in this space also has looked at specifically algorithms for 3D generation. And in that particular case, this one's a little bit different, so I'm happy to tell you about it over coffee.
He spent 2 years digging into every single equation in the generative model specifically for 3D content, where there's this added complication of typically using 2D image generators to create 3D shapes because there just isn't 3D data out there. And kind of answer this question of like, well, the 3D geometry that comes out of those models is way worse than the quality of the 2D images, which maybe feels reasonable because 3D generation is just a hard problem. But then he did kind of an interesting thing, which is he replaced the 3D representation behind the scenes with just a 2D pixel grid.
So, in other words, he made the world's most complicated image generator where you have this procedure for generating content that is used to inform procedure for generating other content, like you had a 2D image generator used to inform making 3D content. You can actually use the same algorithm to use a 2D image generator to inform another for making another 2D image just as a benchmark and found that all the same artifacts actually persisted in that case. So, in other words, it wasn't the fact that you were learning 3D stuff that was the problem, it was the learning procedure itself.
And then he was able to explain the difference, fix the formulas a little bit, and then also improve the 3D case as well. So, anyway, if you like really fine-grained kind of differential equation analysis, I encourage you to take a look. It's a fun paper.
Okay, so in any event, I think hopefully you understand the high-level point, which is that essentially when it comes to engineering these generative models like in diffusion, there's a very complicated story. And this, by the way, is not unique to our research team. There are other ones in this domain. For example, some folks at Flatiron Institute that have come to very similar conclusion. There's some interaction between the geometry of your embedding that, like reshaping your embedding into some nice shape, makes the sampling procedure easier, which is not surprising. The training procedure itself, noise, and so on that are all creating these things to work well even if we don't have a nice underlying theory that can explain generalization at the most broad level. Then really this is just a call for help, you know, we need all of you guys to come up with the interesting sort of theories that explain what's going on.
And I think it's not just a question of theoretical research that us academics can kind of do as a fun byproduct while OpenAI does the real work. I think there's some real promise here for engineering lightweight models that have similar performance to these giant things once we can explain where machine learning is actually necessary in this pipeline. And I don't think we have a good answer for that.
In any event, because I've organized this talk poorly, I'm now going to jump to a completely different topic and tell you a bit about not just navigating these learning embeddings, but also editing them in different fashion. This is work largely from my former postdoc Jiaqing who moved to Meta, as well as my student Rickard Gabrielson who's about to graduate.
And this has to do with taking those matrices in these large models and learning to fine-tune and adjust them for different tasks.
And this is something that we see in industry all the time. I have some generic pre-trained maybe language model, and maybe I have some in-house data set. So, for example, maybe you're a lawyer and you have a bunch of documents that you can't release, and now I want my model to be an expert on that stuff. So, it's just a little bit of additional training to incorporate that data.
And one of the issues there is that those data sets tend to be a lot smaller than the original model. So, maybe it doesn't make sense to train all of the millions or billions of weights in the original guy.
Or maybe you're an academic, you don't have a million GPUs to train on, and you just need some lightweight training procedure.
And so, in practice, there's one procedure that seems to have arisen as a very popular technique, which is very much just kind of built on some engineering practice.
Here I've borrowed a figure from the literature. I think this is so much fun.
Somehow, I guess this particular corner of machine learning is more comfortable with like a pictorial, I don't know, version of a matrix multiple.
But in any event, there's this procedure called low-rank adaptation for fine-tuning neural networks, which is a very straightforward thing. Which is your neural network are these giant matrices which I'm going to call W naught, which are mapping your data into these other spaces followed by some non-linearity like ReLU.
Typically, when you fine-tune, maybe you adjust these W naughts, but for whatever reason, it's too big. The data set's too large or your GPU isn't good enough.
Then, one thing people will do in practice is rather than updating every element of W naught, they will make an additive change of a low-rank matrix and instead try to optimize for these two matrices B and A, which are like skinny and tall.
Right? And this is a way to reduce the number of parameters that you have to learn during the fine-tuning procedure. This is part of a larger literature called parameter efficient fine-tuning, but I think it's actually a pretty sensible one. Like I don't know. As a person that's much more comfortable with numerical linear algebra than with machine learning, I think if you look at this, it's kind of like a reasonable thing. It kind of resembles what we used to do, for example, for the Netflix problem back in the day in matrix completion and so on.
So, low-rank adaptation has been around for a while now and I think in industry is one of the more popular ways that we have for fine-tuning.
And our post-doc Jatin did kind of an interesting thing, which is he dug through the user manuals for this software. We were working with friends at IBM at the time and we were just trying to get a handle on like what people do in this particular domain. And one thing that he noticed by, I don't know, miracle or his own brilliance, I'm sure, was that they all offered the same weird practical advice. So again, what you're going to do is you'll fix these giant matrices W naught in your neural network and you're just going to fine-tune this product B and A. So you now have two different matrices that get updated.
And all of them follow the same general procedure, which is typically when you initialize these fine-tunings, you want the product of A and B to be zero because you want to start out your original model and then learn to perturb it.
And so they all said the same thing, which is initialize B to be zero and A to be random in Gaussian.
Right? Or there's some other pieces of practical advice which are kind of similar where like maybe the learning rate for one of these two matrices is larger than the other, which are things that maybe you wouldn't have guessed just looking at this low-rank adaptation problem at its face.
So Jatin did the completely sensible thing as just like a scientist, which is he said, "Well, that seems odd. Why don't I do the opposite of that? Like why don't I zero out A and have a Gaussian random B?"
And what he found was that performance degraded significantly.
And this is really surprising. I mean, it sounds like essentially we were all reading these user manuals thinking like, "What an odd choice?" But it turns out that this is actually a really critical little piece of the puzzle to make these low-rank adaptations work, which really caught us by surprise.
So then we started stripping away more and more pieces of our neural network, you know, using simpler and simpler models and testing out this fine-tuning procedure and found that this is very much a pattern that gets you, you know, in fact, if you remove all of your non-linearities, whatever, still this practical advice turns out to be the right thing for learning low-rank changes to matrices. And finally, we were in the domain of least squares because we've removed all the non-linearities. So you can actually prove a theorem.
And here's the kind of surprising result. Roughly what you can prove is that you can lower bound the performance of fine-tuning only one of these matrices making the other Gaussian random in terms of the other, right? And the reason you can do this, kind of similarly to what I mentioned for diffusion models, is like there's a closed-form formula for A and B, where you fix one and optimize for the other because this is just least squares.
And so you really can just work out the statistics of this thing.
So at least for linear models, you can justify this procedure. Like really, fine-tuning one of these matrices is more important than the other.
For certain corners of statistics, this is kind of an unsurprising result in retrospect. I think the way that you can view it is there's a lot of results in random matrix theory that say if I have a high-dimensional data set and I just randomly project it onto a low-dimensional space, a lot of times structure is preserved.
And so in some sense, if I just make this A matrix here, by the way, I think my slides keep swapping A and B and for that I apologize. If A is just Gaussian random, it's kind of serving the role of that random projection, which often preserves a lot of structure, whereas B is really critical to the learning process. And so somehow that's what's getting reflected here, just in this new weird non-linear high-dimensional universe that we're in in machine learning.
And so in fact what Jatin did which is kind of fun is he said well in that case we can probably get similar performance with half the parameters by doing the following which is fixing A to be Gaussian random and not fine-tuning it at all.
And if I do that then I don't even need to store A. I can just store the random number seed that I used to generate it. So this is very efficient in terms of parameters.
And in fact here's a useless table. If you're able to read it from a distance then your vision is better than mine. But the main takeaway is that in fact you can get very similar performance to fine-tuning both of these matrices if you design your procedure really carefully. In fact you can get better performance because you can now have two times the number of parameters in the matrix that matters. And so this was a nice kind of practical takeaway that was built on this little theoretical alternative thing.
Now then we got a little bit greedy. So I think one thing that you can learn from this research project is this parameter efficient fine-tuning procedure like this learning a low-rank update is it something that parameter efficient?
