Okay. Hello. Hello everybody. I'll do it like this. So you might be surprised to see me here. I'm not Zanap. So Zanap, it turned out in the end, through various complexities in family etc., couldn't make it. And at the same time I heard some complaints from people to me and say, why don't you speak at the Tübingen summer school? So we thought we can address both issues in one and I'll give you one talk here and it's going to be special, not in terms of the content, well at least I don't want to claim that, but also since you will get not just me but an additional surprise visitor if everything works out in the talk. So don't fall asleep. I'm using this as an excuse to motivate you for my talk, don't fall asleep, something interesting might happen. And the talk, it's actually good that it's in the second week because it builds upon some things that you've already learned about, but it tries to put it in a broad perspective and maybe also connect it a little bit to scientific applications.
So I've called it a tale of spurious correlations. So I think you've all seen this kind of examples. So this is an example where people trained a neural network to detect so-called pneumothorax, where the lung collapses, and they built this system, wanted to apply it to new incoming patients, and it didn't work. And they noticed that it had worked very well in the training set, didn't work on the incoming patients. Then they started to analyze what the system was looking at, and then they noticed that it was looking, for instance, at this area here, which is an area, I don't know if you can see it, where the doctors insert a little tube into the lung of the patients to let the liquid flow out. So this means pneumothorax patients, they come in, they get diagnosed, they get their little tube inserted into the lung, and then they take X-ray pictures, and then they train on those X-ray pictures. Now clearly for the incoming patients, they wanted to do the diagnosis in the first place. So there was no tube in the lung. So we had a different distribution, and so from that point of view, the tube appeared, or it was correlated with the pneumothorax, but it's not causal for it. So this correlation, it appeared in the training set but not in the test set. It was not a robust correlation. So that's one example of something that can go wrong, or one example of something that people call a spurious correlation.
Now, some people would argue, or people, the current scaling trend seems to suggest that we don't have to worry about all these things as long as we scale up our models far enough, they're going to learn about causality magically. So for this reason, I want to show you some images of where you still find spurious correlations in current foundation models. And this is an image of the Ebbinghaus illusion. So if you look at these two orange discs, then you will notice that the left disc looks a little bit bigger than the right disc. And in fact, they are the same. So you can, if you want, you can measure with your fingers. But it's because they're surrounded by other discs. Now suppose you're a foundation model. You're trained on the internet. You're trained also on images, and the images appear with captions, and maybe you've seen some images like this alongside with a caption that explains the illusion. Now you could learn something from this. You could learn different things from it. For instance, you could learn whenever I have orange discs surrounded by blue discs, those orange discs are the same size. Now, that would be a stupid conclusion. It would be a spurious correlation because clearly it's not going to be robust. And if you test it on images like this, and this is an example that two of our students have come up with, test it on images like this, then if you fall for the spurious correlation, then you will think these orange discs are the same size. And in fact, this is what language models tend to do. So here it says both orange circles are actually the same size, and it even recognizes the correct illusion, the Ebbinghaus illusion. And I found this a lot of fun and tried out multiple illusions. So maybe you've seen this illusion, so this length is the same as this length, although they look different. It's related to a certain prior in 3D perception, or this is a straight line even though it doesn't look like it. Now let's exaggerate those illusions as well. So this is an extreme version where the two segments, so this segment and this segment are clearly not the same length. But again it says this is the Müller-Lyer illusion, those are the same length, or here it says this is actually a straight line. And finally this one is also grossly exaggerated. They're clearly not parallel. It says they are parallel. I showed it to someone last week and they were asking, well you were using 40, what about the more recent model? So this morning I ran it on Fable and he says, yeah, this looks like the cylinder illusion. And so what's actually on the image? Where does it say anyway? So I don't want to read the whole thing, but it even gives some explanation what it feels like when you look at this, etc. But it still falls for this kind of spurious correlations.
Now, spurious correlations, they sound like a bad thing, but if you model them, if you understand them, they can also be a good thing. And that's an example from astronomy. And I'm going to use this as an excuse to show you several examples from astronomy, several examples of machine learning in astronomy. And maybe just to connect this also a little bit to what Jeb Batista was saying in the first week. So it's, I agree, it's non-trivial. If you think all of AI now is about who builds the best language model, then it's non-trivial to compete with companies that have millions of GPUs. But AI is much broader. AI is also about what are the future algorithms, and AI currently and very excitingly is about what you can do in the sciences by using AI. So it's like in astronomy when the telescope was discovered, you could get a telescope and make new discovery, or when the telescope was around, you could make lots of new astronomical discoveries just by applying the telescope. And I think we're a little bit in the situation now. So here we're going to apply it to the problem of exoplanet search, and we will do this on... it's easier. Okay. So this is actually a good time for our surprise visit before I get into this topic, and we're going to switch the screen now and have maybe 15 minutes of a conversation with our surprise guest. And let's see, let's see if this works. Hello. Can you see me?
We can hear you already. Now we can see you. So I would like to introduce to the audience, this is Arthur Mensch, whom we had been in contact about whether he can join us for a little while here. And the excuse for doing so was that he's an MLSS Tübingen alumnus. He was here before, sitting on your side of the room, different building at the time. So we're very happy that he's here now, and he's one of the people really moving European AI also at the frontier. So Arthur, maybe we can first start and you can tell us a little bit about your experience when you were an MLSS student. What do you remember?
